Dynamic readout of behaviorally relevant signals from area MT during task switching

Inferring context-dependent computations through linear approximations of prefrontal cortex dynamics

The complex neural activity of prefrontal cortex (PFC) is a hallmark of cognitive processes. How these rich dynamics emerge and support neural computations is largely unknown. Here, we infer mechanisms underlying the context-dependent integration of sensory inputs by fitting dynamical models to PFC population responses of behaving monkeys. A class of models implementing linear dynamics driven by external inputs accurately captured PFC responses within contexts and revealed equally performing mechanisms. One model implemented context-dependent recurrent dynamics and relied on transient input amplification; the other relied on subtle contextual modulations of the inputs, providing constraints on the attentional effects in sensory areas required to explain flexible PFC responses and behavior. Both models revealed properties of inputs and recurrent dynamics that were not apparent from qualitative descriptions of PFC responses. By revealing mechanisms that are quantitatively consistent with complex cortical dynamics, our modeling approach provides a principled and general framework to link neural population activity and computation.

Decision-related activity and movement selection in primate visual cortex

Fluctuations in the activity of sensory neurons often predict perceptual decisions. This connection can be quantified with a metric called choice probability (CP), and there is a longstanding debate about whether CP reflects a causal influence on decisions or an echo of decision-making activity elsewhere in the brain. Here, we show that CP can reflect a third variable, namely, the movement used to indicate the decision. In a standard visual motion discrimination task, neurons in the middle temporal (MT) area of primate cortex responded more strongly during trials that involved a saccade toward their receptive fields. This variability accounted for much of the CP observed across the neuronal population, and it arose through training. Moreover, pharmacological inactivation of MT biased behavioral responses away from the corresponding visual field locations. These results demonstrate that training on a task with fixed sensorimotor contingencies introduces movement-related activity in sensory brain regions and that this plasticity can shape the neural circuitry of perceptual decision-making.

Employment of time-varying sensory evidence to test the mechanisms underlying flexible decision-making

To make flexible decisions in dynamic environments, the brain must integrate behaviorally relevant information while simultaneously discarding irrelevant information. This study aimed to investigate the mechanisms responsible for discarding irrelevant information during context-dependent decision-making. We trained two macaque monkeys to switch between direction and depth discrimination tasks in successive trials. During decision-making, the strength of the motion or depth signal changes transiently at various times, introducing a brief pulse. We assessed the effects of pulse on behavioral choices. Consistent with previous findings, early relevant pulses, such as motion pulses during direction discrimination, had a significantly larger effect compared to late pulses. Critically, the effects of irrelevant pulses, such as motion pulses during depth discrimination, exhibited an initial minimal effect, followed by an increase and subsequent decrease as a function of pulse timing. Gating mechanisms alone, aimed at discarding irrelevant information, did not account for the observed time course of pulse effects. Instead, the observed increase in the effects of irrelevant pulses with time suggested the involvement of a leaky integration mechanism. The results suggested that the brain controls the amount of disposal in accumulating sensory evidence during flexible decision-making.

Early selection of task-relevant features through population gating

Brains can gracefully weed out irrelevant stimuli to guide behavior. This feat is believed to rely on a progressive selection of task-relevant stimuli across the cortical hierarchy, but the specific across-area interactions enabling stimulus selection are still unclear. Here, we propose that population gating, occurring within primary auditory cortex (A1) but controlled by top-down inputs from prelimbic region of medial prefrontal cortex (mPFC), can support across-area stimulus selection. Examining single-unit activity recorded while rats performed an auditory context-dependent task, we found that A1 encoded relevant and irrelevant stimuli along a common dimension of its neural space. Yet, the relevant stimulus encoding was enhanced along an extra dimension. In turn, mPFC encoded only the stimulus relevant to the ongoing context. To identify candidate mechanisms for stimulus selection within A1, we reverse-engineered low-rank RNNs trained on a similar task. Our analyses predicted that two context-modulated neural populations gated their preferred stimulus in opposite contexts, which we confirmed in further analyses of A1. Finally, we show in a two-region RNN how population gating within A1 could be controlled by top-down inputs from PFC, enabling flexible across-area communication despite fixed inter-areal connectivity.

Neural Mechanisms That Make Perceptual Decisions Flexible

Neural mechanisms of perceptual decision making have been extensively studied in experimental settings that mimic stable environments with repeating stimuli, fixed rules, and payoffs. In contrast, we live in an ever-changing environment and have varying goals and behavioral demands. To accommodate variability, our brain flexibly adjusts decision-making processes depending on context. Here, we review a growing body of research that explores the neural mechanisms underlying this flexibility. We highlight diverse forms of context dependency in decision making implemented through a variety of neural computations. Context-dependent neural activity is observed in a distributed network of brain structures, including posterior parietal, sensory, motor, and subcortical regions, as well as the prefrontal areas classically implicated in cognitive control. We propose that investigating the distributed network underlying flexible decisions is key to advancing our understanding and discuss a path forward for experimental and theoretical investigations.

A neural correlate of perceptual segmentation in macaque middle temporal cortical area

High-resolution vision requires fine retinal sampling followed by integration to recover object properties. Importantly, accuracy is lost if local samples from different objects are intermixed. Thus, segmentation, grouping of image regions for separate processing, is crucial for perception. Previous work has used bi-stable plaid patterns, which can be perceived as either a single or multiple moving surfaces, to study this process. Here, we report a relationship between activity in a mid-level site in the primate visual pathways and segmentation judgments. Specifically, we find that direction selective middle temporal neurons are sensitive to texturing cues used to bias the perception of bi-stable plaids and exhibit a significant trial-by-trial correlation with subjective perception of a constant stimulus. This correlation is greater in units that signal global motion in patterns with multiple local orientations. Thus, we conclude the middle temporal area contains a signal for segmenting complex scenes into constituent objects and surfaces.

The NMDA receptor antagonist ketamine impairs and delays context-dependent decision making in the parietal cortex

Flexible decision making is an indispensable ability for humans. A subanesthetic dose of ketamine, an N-methyl-D-aspartate receptor antagonist, impairs this flexibility in a manner that is similar to patients with schizophrenia; however how it affects neural processes related to decision making remains unclear. Here, we report that ketamine administration impairs neural processing related to context-dependent decision making, and delays the onset of decision making. We recorded single unit activity in the lateral intraparietal area (LIP) while monkeys switched between a direction-discrimination task and a depth-discrimination task. Ketamine impaired choice accuracy for incongruent stimuli that required different decisions depending on the task, for the direction-discrimination task. Neural sensitivity to irrelevant depth information increased with ketamine during direction discrimination in LIP, indicating impaired processing of irrelevant information. Furthermore, the onset of decision-related neural activity was delayed in conjunction with an increased reaction time irrespective of task and stimulus congruency. Neural sensitivity and response onset of the middle temporal area (MT) were not modulated by ketamine, indicating that ketamine worked on neural decision processes downstream of MT. These results suggest that ketamine administration may impair what information to process and when to process it for the purpose of achieving flexible decision making.

Task-Specific Employment of Sensory Signals Underlies Rapid Task Switching

Much of our flexible behavior is dependent on responding efficiently to relevant information while discarding irrelevant information. Little is known, however, about how neural pathways governing sensory-motor associations can rapidly switch to accomplish such flexibility. Here, we addressed this question by electrically microstimulating middle temporal (MT) neurons selective for both motion direction and binocular disparity in monkeys switching between direction and depth discrimination tasks. Surprisingly, we frequently found that the observed psychophysical bias precipitated by delivering microstimulation to neurons whose preferred direction and depth were related to opposite choices in the two tasks was substantially shifted toward a specific movement. Furthermore, these effects correlated with behavioral switching performance. Our findings suggest that the outputs of sensory signals are task specific and that irrelevant sensory-motor pathways are gated depending on task demand so as to accomplish rapid attentional switching.

Perceptual learning as a result of concerted changes in prefrontal and visual cortex

Our perceptual ability remarkably improves with training. Some studies on visual perceptual learning have shown refined neural representation of the trained stimulus in the visual cortex, whereas others have exclusively argued for improved readout and decision-making processes in the frontoparietal cortex. The mixed results have rendered the underlying neural mechanisms puzzling and hotly debated. By simultaneously recording from monkey visual area V4 and ventrolateral prefrontal cortex (PFC) implanted with microelectrode arrays, we dissected learning-induced cortical changes over the course of training the monkeys in a global form detection task. Decoding analysis dissociated two distinct components of neuronal population codes that were progressively and markedly enhanced in both V4 and PFC. One component was closely related to the target stimulus feature and was subject to task-dependent top-down modulation; it emerged earlier in V4 than PFC and its enhancement was specific to the trained configuration of the target stimulus. The other component of the neural code was entirely related to the animal's behavioral choice; it emerged earlier in PFC than V4 and its enhancement completely generalized to an untrained stimulus configuration. These results implicate two concurrent and synergistic learning processes: a perceptual process that is specific to the details of the trained stimulus feature and a cognitive process that is dependent on the total amount of learning experience in the trained task. When combined, these two learning processes were well predictive of the animal's learning behavior.

Choice-related activity and neural encoding in primary auditory cortex and lateral belt during feature-selective attention

Selective attention is necessary to sift through, form a coherent percept of, and make behavioral decisions on the vast amount of information present in most sensory environments. How and where selective attention is employed in cortex and how this perceptual information then informs the relevant behavioral decisions is still not well understood. Studies probing selective attention and decision-making in visual cortex have been enlightening as to how sensory attention might work in that modality; whether or not similar mechanisms are employed in auditory attention is not yet clear. Therefore, we trained rhesus macaques on a feature-selective attention task, where they switched between reporting changes in temporal (amplitude modulation, AM) and spectral (carrier bandwidth) features of a broadband noise stimulus. We investigated how the encoding of these features by single neurons in primary (A1) and secondary (middle lateral belt, ML) auditory cortex was affected by the different attention conditions. We found that neurons in A1 and ML showed mixed selectivity to the sound and task features. We found no difference in AM encoding between the attention conditions. We found that choice-related activity in both A1 and ML neurons shifts between attentional conditions. This finding suggests that choice-related activity in auditory cortex does not simply reflect motor preparation or action and supports the relationship between reported choice-related activity and the decision and perceptual process.NEW & NOTEWORTHY We recorded from primary and secondary auditory cortex while monkeys performed a nonspatial feature attention task. Both areas exhibited rate-based choice-related activity. The manifestation of choice-related activity was attention dependent, suggesting that choice-related activity in auditory cortex does not simply reflect arousal or motor influences but relates to the specific perceptual choice.

Amplitude modulations of cortical sensory responses in pulsatile evidence accumulation

How does the brain internally represent a sequence of sensory information that jointly drives a decision-making behavior? Studies of perceptual decision-making have often assumed that sensory cortices provide noisy but otherwise veridical sensory inputs to downstream processes that accumulate and drive decisions. However, sensory processing in even the earliest sensory cortices can be systematically modified by various external and internal contexts. We recorded from neuronal populations across posterior cortex as mice performed a navigational decision-making task based on accumulating randomly timed pulses of visual evidence. Even in V1, only a small fraction of active neurons had sensory-like responses time-locked to each pulse. Here, we focus on how these 'cue-locked' neurons exhibited a variety of amplitude modulations from sensory to cognitive, notably by choice and accumulated evidence. These task-related modulations affected a large fraction of cue-locked neurons across posterior cortex, suggesting that future models of behavior should account for such influences.

Rules warp feature encoding in decision-making circuits

We have the capacity to follow arbitrary stimulus-response rules, meaning simple policies that guide our behavior. Rule identity is broadly encoded across decision-making circuits, but there are less data on how rules shape the computations that lead to choices. One idea is that rules could simplify these computations. When we follow a rule, there is no need to encode or compute information that is irrelevant to the current rule, which could reduce the metabolic or energetic demands of decision-making. However, it is not clear if the brain can actually take advantage of this computational simplicity. To test this idea, we recorded from neurons in 3 regions linked to decision-making, the orbitofrontal cortex (OFC), ventral striatum (VS), and dorsal striatum (DS), while macaques performed a rule-based decision-making task. Rule-based decisions were identified via modeling rules as the latent causes of decisions. This left us with a set of physically identical choices that maximized reward and information, but could not be explained by simple stimulus-response rules. Contrasting rule-based choices with these residual choices revealed that following rules (1) decreased the energetic cost of decision-making; and (2) expanded rule-relevant coding dimensions and compressed rule-irrelevant ones. Together, these results suggest that we use rules, in part, because they reduce the costs of decision-making through a distributed representational warping in decision-making circuits.

A Biased Bayesian Inference for Decision-Making and Cognitive Control

Although classical decision-making studies have assumed that subjects behave in a Bayes-optimal way, the sub-optimality that causes biases in decision-making is currently under debate. Here, we propose a synthesis based on exponentially-biased Bayesian inference, including various decision-making and probability judgments with different bias levels. We arrange three major parameter estimation methods in a two-dimensional bias parameter space (prior and likelihood), of the biased Bayesian inference. Then, we discuss a neural implementation of the biased Bayesian inference on the basis of changes in weights in neural connections, which we regarded as a combination of leaky/unstable neural integrator and probabilistic population coding. Finally, we discuss mechanisms of cognitive control which may regulate the bias levels.

Probing Sensory Readout via Combined Choice-Correlation Measures and Microstimulation Perturbation

It is controversial whether covariation between neuronal activity and perceptual choice (i.e., choice correlation) reflects the functional readout of sensory signals. Here, we combined choice-correlation measures and electrical microstimulation on a site-to-site basis in the medial superior temporal area (MST), middle temporal area (MT), and ventral intraparietal area (VIP) when macaques discriminated between motion directions in both fine and coarse tasks. Microstimulation generated comparable effects between tasks but heterogeneous effects across and within brain regions. Within the MST and MT, microstimulation significantly biased an animal's choice toward the sensory preference instead of choice-related signals of the stimulated units. This was particularly evident for sites with conflict preference of sensory and choice-related signals. In the VIP, microstimulation failed to produce significant effects in either task despite strong choice correlations presented in this area. Our results suggest that sensory readout may not be inferred from choice-related signals during perceptual decision-making tasks.

Perceptual Decisions in the Presence of Relevant and Irrelevant Sensory Evidence

Perceptual decisions in the presence of decision-irrelevant sensory information require a selection of decision-relevant sensory evidence. To characterize the mechanism that is responsible for separating decision-relevant from irrelevant sensory information we asked human subjects to make judgments about one of two simultaneously present motion components in a random dot stimulus. Subjects were able to ignore the decision-irrelevant component to a large degree, but their decisions were still influenced by the irrelevant sensory information. Computational modeling revealed that this influence was not simply the consequence of subjects forgetting at times which stimulus component they had been instructed to base their decision on. Instead, residual irrelevant information always seems to be leaking through, and the decision process is captured by a net sensory evidence signal being accumulated to a decision threshold. This net sensory evidence is a linear combination of decision-relevant and irrelevant sensory information. The selection process is therefore well-described by a strong linear gain modulation, which, in our experiment, resulted in the relevant sensory evidence having at least 10 times more impact on the decision than the irrelevant evidence.

Decoupled choice-driven and stimulus-related activity in parietal neurons may be misrepresented by choice probabilities

Trial-by-trial correlations between neural responses and choices (choice probabilities) are often interpreted to reflect a causal contribution of neurons to task performance. However, choice probabilities may arise from top-down, rather than bottom-up, signals. We isolated distinct sensory and decision contributions to single-unit activity recorded from the dorsal medial superior temporal (MSTd) and ventral intraparietal (VIP) areas of monkeys during perception of self-motion. Superficially, neurons in both areas show similar tuning curves during task performance. However, tuning in MSTd neurons primarily reflects sensory inputs, whereas choice-related signals dominate tuning in VIP neurons. Importantly, the choice-related activity of VIP neurons is not predictable from their stimulus tuning, and these factors are often confounded in choice probability measurements. This finding was confirmed in a subset of neurons for which stimulus tuning was measured during passive fixation. Our findings reveal decoupled stimulus and choice signals in the VIP area, and challenge our understanding of choice signals in the brain.Choice-related signals in neuronal activity may reflect bottom-up sensory processes, top-down decision-related influences, or a combination of the two. Here the authors report that choice-related activity in VIP neurons is not predictable from their stimulus tuning, and that dominant choice signals can bias the standard metric of choice preference (choice probability).

Defining the V5/MT neuronal pool for perceptual decisions in a visual stereo-motion task

In the primate visual cortex, neurons signal differences in the appearance of objects with high precision. However, not all activated neurons contribute directly to perception. We defined the perceptual pool in extrastriate visual area V5/MT for a stereo-motion task, based on trial-by-trial co-variation between perceptual decisions and neuronal firing (choice probability (CP)). Macaque monkeys were trained to discriminate the direction of rotation of a cylinder, using the binocular depth between the moving dots that form its front and rear surfaces. We manipulated the activity of single neurons trial-to-trial by introducing task-irrelevant stimulus changes: dot motion in cylinders was aligned with neuronal preference on only half the trials, so that neurons were strongly activated with high firing rates on some trials and considerably less activated on others. We show that single neurons maintain high neurometric sensitivity for binocular depth in the face of substantial changes in firing rate. CP was correlated with neurometric sensitivity, not level of activation. In contrast, for individual neurons, the correlation between perceptual choice and neuronal activity may be fundamentally different when responding to different stimulus versions. Therefore, neuronal pools supporting sensory discrimination must be structured flexibly and independently for each stimulus configuration to be discriminated.

This article is part of the themed issue ‘Vision in our three-dimensional world'.

Context-Dependent Accumulation of Sensory Evidence in the Parietal Cortex Underlies Flexible Task Switching

Switching behavior based on multiple rules is a fundamental ability of flexible behavior. Although interactions among the frontal, parietal, and sensory cortices are necessary for such flexibility, little is known about the neural computations concerning context-dependent information readouts. Here, we provide evidence that neurons in the lateral intraparietal area (LIP) accumulate relevant information preferentially depending on context. We trained monkeys to switch between direction and depth discrimination tasks and analyzed the buildup activity in the LIP depending on task context. In accordance with behavior, the rate of buildup to identical visual stimuli differed between tasks and buildup was prominent only for the stimulus dimension relevant to the task. These results indicate that LIP neurons accumulate relevant information depending on context to decide flexibly where to move the eye, suggesting that flexibility is, at least partly, implemented in the form of temporal integration gain control.

SIGNIFICANCE STATEMENT

Flexible behavior depending on context is a hallmark of human cognition. During flexible behavior, the frontal and parietal cortices have complex representations that hinder efforts to conceptualize their underlying computations. We now provide evidence that neurons in the lateral intraparietal area accumulate relevant information preferentially depending on context. We suggest that behavioral flexibility is implemented in the form of temporal integration gain control in the parietal cortex.

Task-dependent recurrent dynamics in visual cortex

The capacity for flexible sensory-action association in animals has been related to context-dependent attractor dynamics outside the sensory cortices. Here, we report a line of evidence that flexibly modulated attractor dynamics during task switching are already present in the higher visual cortex in macaque monkeys. With a nonlinear decoding approach, we can extract the particular aspect of the neural population response that reflects the task-induced emergence of bistable attractor dynamics in a neural population, which could be obscured by standard unsupervised dimensionality reductions such as PCA. The dynamical modulation selectively increases the information relevant to task demands, indicating that such modulation is beneficial for perceptual decisions. A computational model that features nonlinear recurrent interaction among neurons with a task-dependent background input replicates the key properties observed in the experimental data. These results suggest that the context-dependent attractor dynamics involving the sensory cortex can underlie flexible perceptual abilities.

Feature-Selective Attention Adaptively Shifts Noise Correlations in Primary Auditory Cortex

Sensory environments often contain an overwhelming amount of information, with both relevant and irrelevant information competing for neural resources. Feature attention mediates this competition by selecting the sensory features needed to form a coherent percept. How attention affects the activity of populations of neurons to support this process is poorly understood because population coding is typically studied through simulations in which one sensory feature is encoded without competition. Therefore, to study the effects of feature attention on population-based neural coding, investigations must be extended to include stimuli with both relevant and irrelevant features. We measured noise correlations (r_noise) within small neural populations in primary auditory cortex while rhesus macaques performed a novel feature-selective attention task. We found that the effect of feature-selective attention on r_noise depended not only on the population tuning to the attended feature, but also on the tuning to the distractor feature. To attempt to explain how these observed effects might support enhanced perceptual performance, we propose an extension of a simple and influential model in which shifts in r_noise can simultaneously enhance the representation of the attended feature while suppressing the distractor. These findings present a novel mechanism by which attention modulates neural populations to support sensory processing in cluttered environments.SIGNIFICANCE STATEMENT Although feature-selective attention constitutes one of the building blocks of listening in natural environments, its neural bases remain obscure. To address this, we developed a novel auditory feature-selective attention task and measured noise correlations (r_noise) in rhesus macaque A1 during task performance. Unlike previous studies showing that the effect of attention on r_noise depends on population tuning to the attended feature, we show that the effect of attention depends on the tuning to the distractor feature as well. We suggest that these effects represent an efficient process by which sensory cortex simultaneously enhances relevant information and suppresses irrelevant information.

Insights into decision making using choice probability

A long-standing question in systems neuroscience is how the activity of single neurons gives rise to our perceptions and actions. Critical insights into this question occurred in the last part of the 20th century when scientists began linking modulations of neuronal activity directly to perceptual behavior. A significant conceptual advance was the application of signal detection theory to both neuronal activity and behavior, providing a quantitative assessment of the relationship between brain and behavior. One metric that emerged from these efforts was choice probability (CP), which provides information about how well an ideal observer can predict the choice an animal makes from a neuron's discharge rate distribution. In this review, we describe where CP has been studied, locational trends in the values found, and why CP values are typically so low. We discuss its dependence on correlated activity among neurons of a population, assess whether it arises from feedforward or feedback mechanisms, and investigate what CP tells us about how many neurons are required for a decision and how they are pooled to do so.

A Modality-Specific Feedforward Component of Choice-Related Activity in MT

The activity of individual sensory neurons can be predictive of an animal's choices. These decision signals arise from network properties dependent on feedforward and feedback inputs; however, the relative contributions of these inputs are poorly understood. We determined the role of feedforward pathways to decision signals in MT by recording neuronal activity while monkeys performed motion and depth tasks. During each session, we reversibly inactivated V2 and V3, which provide feedforward input to MT that conveys more information about depth than motion. We thus monitored the choice-related activity of the same neuron both before and during V2/V3 inactivation. During inactivation, MT neurons became less predictive of decisions for the depth task but not the motion task, indicating that a feedforward pathway that gives rise to tuning preferences also contributes to decision signals. We show that our data are consistent with V2/V3 input conferring structured noise correlations onto the MT population.

Sensory integration dynamics in a hierarchical network explains choice probabilities in cortical area MT

Neuronal variability in sensory cortex predicts perceptual decisions. This relationship, termed choice probability (CP), can arise from sensory variability biasing behaviour and from top-down signals reflecting behaviour. To investigate the interaction of these mechanisms during the decision-making process, we use a hierarchical network model composed of reciprocally connected sensory and integration circuits. Consistent with monkey behaviour in a fixed-duration motion discrimination task, the model integrates sensory evidence transiently, giving rise to a decaying bottom-up CP component. However, the dynamics of the hierarchical loop recruits a concurrently rising top-down component, resulting in sustained CP. We compute the CP time-course of neurons in the medial temporal area (MT) and find an early transient component and a separate late contribution reflecting decision build-up. The stability of individual CPs and the dynamics of noise correlations further support this decomposition. Our model provides a unified understanding of the circuit dynamics linking neural and behavioural variability.

Neural correlates of task switching in prefrontal cortex and primary auditory cortex in a novel stimulus selection task for rodents

Animals can selectively respond to a target sound despite simultaneous distractors, just as humans can respond to one voice at a crowded cocktail party. To investigate the underlying neural mechanisms, we recorded single-unit activity in primary auditory cortex (A1) and medial prefrontal cortex (mPFC) of rats selectively responding to a target sound from a mixture. We found that prestimulus activity in mPFC encoded the selection rule-which sound from the mixture the rat should select. Moreover, electrically disrupting mPFC significantly impaired performance. Surprisingly, prestimulus activity in A1 also encoded selection rule, a cognitive variable typically considered the domain of prefrontal regions. Prestimulus changes correlated with stimulus-evoked changes, but stimulus tuning was not strongly affected. We suggest a model in which anticipatory activation of a specific network of neurons underlies the selection of a sound from a mixture, giving rise to robust and widespread rule encoding in both brain regions.

Visual impairment by surrounding noise is due to interactions among stimuli in the higher-order visual cortex

Observers have difficulty identifying a target in their peripheral vision in the presence of surrounding stimuli. Although hypotheses addressing this phenomenon have been proposed, such as the integration of stimuli and surround suppression in the higher-order visual cortex, no direct comparisons of the psychophysical and neuronal sensitivities have been performed. Here we measured the performance of monkeys with a variant of the direction discrimination task using a center/surround bipartite random-dot stimulus while simultaneously recording from isolated neurons from the middle temporal visual area (MT). The psychophysical threshold increased with the addition of a task-irrelevant noise annulus that surrounded the task-relevant motion stimuli. The neuronal threshold of MT neurons also increased at a spatial scale similar to the psychophysical threshold. This suggests that the impaired ability in our task resulted from impairment in the MT area. Importantly, reduced neuronal performance was due to both a reduced response to preferred motion and an enhanced response to nonpreferred motion. These observations suggest that impairment caused by surrounding noise results from interactions between stimuli and noise and not from a reduction in the response of visual neurons.

A tweaking principle for executive control: neuronal circuit mechanism for rule-based task switching and conflict resolution

A hallmark of executive control is the brain's agility to shift between different tasks depending on the behavioral rule currently in play. In this work, we propose a "tweaking hypothesis" for task switching: a weak rule signal provides a small bias that is dramatically amplified by reverberating attractor dynamics in neural circuits for stimulus categorization and action selection, leading to an all-or-none reconfiguration of sensory-motor mapping. Based on this principle, we developed a biologically realistic model with multiple modules for task switching. We found that the model quantitatively accounts for complex task switching behavior: switch cost, congruency effect, and task-response interaction; as well as monkey's single-neuron activity associated with task switching. The model yields several testable predictions, in particular, that category-selective neurons play a key role in resolving sensory-motor conflict. This work represents a neural circuit model for task switching and sheds insights in the brain mechanism of a fundamental cognitive capability.

Context-dependent computation by recurrent dynamics in prefrontal cortex

Prefrontal cortex is thought to have a fundamental role in flexible, context-dependent behaviour, but the exact nature of the computations underlying this role remains largely unknown. In particular, individual prefrontal neurons often generate remarkably complex responses that defy deep understanding of their contribution to behaviour. Here we study prefrontal cortex activity in macaque monkeys trained to flexibly select and integrate noisy sensory inputs towards a choice. We find that the observed complexity and functional roles of single neurons are readily understood in the framework of a dynamical process unfolding at the level of the population. The population dynamics can be reproduced by a trained recurrent neural network, which suggests a previously unknown mechanism for selection and integration of task-relevant inputs. This mechanism indicates that selection and integration are two aspects of a single dynamical process unfolding within the same prefrontal circuits, and potentially provides a novel, general framework for understanding context-dependent computations.

Joint tuning for direction of motion and binocular disparity in macaque MT is largely separable

Neurons in sensory cortical areas are tuned to multiple dimensions, or features, of their sensory space. Understanding how single neurons represent multiple features is of great interest for determining the informative dimensions of the neurons' response, the decoding algorithms appropriate for extracting this information from the neuronal population, and for determining where specific transformations occur along the visual hierarchy. Despite the established role of cortical area MT in judgments of motion and depth, it is not known how individual neurons jointly encode the two dimensions. We investigated the joint tuning of individual MT neurons for two visual features: direction of motion and binocular disparity, an important depth cue. We found that a separable, multiplicative combination of tuning for the two features can account for more than 90% of the variance in the joint tuning function for over 91% of MT neurons. These results suggest 1) that each feature can be read out independently from MT by simply averaging across the population without regard to the other feature and 2) that the inseparable representations seen in subsequent areas, such as MST, must be computed beyond MT. Intriguingly, we found that the remaining nonseparable component of the joint tuning function often manifested as small but systematic changes in the neurons' preferences for one feature as the other one was varied. We believe this reflects the local columnar organization of tuning for direction and binocular disparity in MT, indicating that joint tuning may provide a new tool with which to probe functional architecture.

A causal role for V5/MT neurons coding motion-disparity conjunctions in resolving perceptual ambiguity

Judgments about the perceptual appearance of visual objects require the combination of multiple parameters, like location, direction, color, speed, and depth. Our understanding of perceptual judgments has been greatly informed by studies of ambiguous figures, which take on different appearances depending upon the brain state of the observer. Here we probe the neural mechanisms hypothesized as responsible for judging the apparent direction of rotation of ambiguous structure from motion (SFM) stimuli. Resolving the rotation direction of SFM cylinders requires the conjoint decoding of direction of motion and binocular depth signals [1, 2]. Within cortical visual area V5/MT of two macaque monkeys, we applied electrical stimulation at sites with consistent multiunit tuning to combinations of binocular depth and direction of motion, while the monkey made perceptual decisions about the rotation of SFM stimuli. For both ambiguous and unambiguous SFM figures, rotation judgments shifted as if we had added a specific conjunction of disparity and motion signals to the stimulus elements. This is the first causal demonstration that the activity of neurons in V5/MT contributes directly to the perception of SFM stimuli and by implication to decoding the specific conjunction of disparity and motion, the two different visual cues whose combination drives the perceptual judgment.

•

Targeted search for brain sites encoding conjunctions of stereo and motion

•

Electrical microstimulation alters perceptual choice and resolves visual ambiguity

•

Causal evidence for direct role of V5/MT in cue combination

•

Stimulation at cortical sites with conjoint encoding is particularly effective

How mechanisms of perceptual decision-making affect the psychometric function

Psychometric functions are often interpreted in the context of Signal Detection Theory, which emphasizes a distinction between sensory processing and non-sensory decision rules in the brain. This framework has helped to relate perceptual sensitivity to the "neurometric" sensitivity of sensory-driven neural activity. However, perceptual sensitivity, as interpreted via Signal Detection Theory, is based on not just how the brain represents relevant sensory information, but also how that information is read out to form the decision variable to which the decision rule is applied. Here we discuss recent advances in our understanding of this readout process and describe its effects on the psychometric function. In particular, we show that particular aspects of the readout process can have specific, identifiable effects on the threshold, slope, upper asymptote, time dependence, and choice dependence of psychometric functions. To illustrate these points, we emphasize studies of perceptual learning that have identified changes in the readout process that can lead to changes in these aspects of the psychometric function. We also discuss methods that have been used to distinguish contributions of the sensory representation versus its readout to psychophysical performance.

A leaky-integrator model as a control mechanism underlying flexible decision making during task switching

The ability to switch between tasks is critical for animals to behave according to context. Although the association between the prefrontal cortex and task switching has been well documented, the ultimate modulation of sensory–motor associations has yet to be determined. Here, we modeled the results of a previous study showing that task switching can be accomplished by communication from distinct populations of sensory neurons. We proposed a leaky-integrator model where relevant and irrelevant information were stored separately in two integrators and task switching was achieved by leaking information from the irrelevant integrator. The model successfully explained both the behavioral and neuronal data. Additionally, the leaky-integrator model showed better performance than an alternative model, where irrelevant information was discarded by decreasing the weight on irrelevant information, when animals initially failed to commit to a task. Overall, we propose that flexible switching is, in part, achieved by actively controlling the amount of leak of relevant and irrelevant information.

Potential confounds in estimating trial-to-trial correlations between neuronal response and behavior using choice probabilities

Correlations between trial-to-trial fluctuations in the responses of individual sensory neurons and perceptual reports, commonly quantified with choice probability (CP), have been widely used as an important tool for assessing the contributions of neurons to behavior. These correlations are usually weak and often require a large number of trials for a reliable estimate. Therefore, working with measures such as CP warrants care in data analysis as well as rigorous controls during data collection. Here we identify potential confounds that can arise in data analysis and lead to biased estimates of CP, and suggest methods to avoid the bias. In particular, we show that the common practice of combining neuronal responses across different stimulus conditions with z-score normalization can result in an underestimation of CP when the ratio of the numbers of trials for the two behavioral response categories differs across the stimulus conditions. We also discuss the effects of using variable time intervals for quantifying neuronal response on CP measurements. Finally, we demonstrate that serious artifacts can arise in reaction time tasks that use varying measurement intervals if the mean neuronal response and mean behavioral performance vary over time within trials. To emphasize the importance of addressing these concerns in neurophysiological data, we present a set of data collected from V1 cells in macaque monkeys while the animals performed a detection task.

Decision-related activity in sensory neurons: correlations among neurons and with behavior

Neurons in early sensory cortex show weak but systematic correlations with perceptual decisions when trained animals perform at psychophysical threshold. These correlations are observed across repeated presentations of identical stimuli and cannot be explained by variation in external factors. The relationship between the activity of individual sensory neurons and the animal's behavioral choice means that even neurons in early sensory cortex carry information about an upcoming decision. This relationship, termed choice probability, may reflect the effect of fluctuations in neuronal firing rate on the animal's decision, but it can also reflect modulation of sensory responses by cognitive factors, or network properties such as variability that is shared among populations of neurons. Here, we review recent work clarifying the relationship among fluctuations in the responses of individual neurons, correlated variability, and behavior in a variety of tasks and cortical areas. We also discuss the possibility that choice probability may in part reflect the influence of cognitive factors on sensory neurons and explore the situations in which choice probability can be used to make inferences about the role of particular sensory neurons in the decision-making process.

The functional link between area MT neural fluctuations and detection of a brief motion stimulus

Fluctuations of neural firing rates in visual cortex are known to be correlated with variations in perceptual performance. It is important to know whether these fluctuations are functionally linked to perception in a causal manner or instead reflect non-causal processes that arise after the perceptual decision is made. We recorded from middle temporal (MT) neurons from monkey subjects while they detected the random occurrence of a brief 50 ms motion pulse that occurred in either of two (or simultaneously in both) random dot patches located in the same hemisphere. The receptive field parameters of the motion pulse were matched to that preferred by each MT neuron under study. This task contained uncertainty in both space and time because, on any given trial, the subjects did not know which patch would contain the motion pulse or when the motion pulse would occur. Covariations between MT activity and behavior began just before the motion pulse onset and peaked at the maximum neural response. These neural-behavioral covariations were strongest when only one patch contained the motion pulse and were still weakly present when a patch did not contain a motion pulse. A feedforward temporal integration model with two independent detector channels captured both the detection performance and evolution of the neural-behavior covariations over time and stimulus condition. The results suggest that, when detecting a brief visual stimulus, there is a causal relationship between fluctuations in neural activity and variations in behavior across trials.

Correlations between the activity of sensory neurons and behavior: how much do they tell us about a neuron's causality?

How the activity of sensory neurons elicits perceptions and guides behavior is central to our understanding of the brain and is a subject of intense investigation in neuroscience. Correlations between the activity of sensory neurons and behavior have been widely observed and are sometimes used to infer how neurons are used to guide a certain behavior. This view is challenged firstly by theoretical considerations that these correlations rely on the existence of correlated noise and its structure, and secondly by recent empirical observations suggesting that such correlated noise is not a fixed network property but that it depends on various sources, and varies with a subject's mental state.

A neuronal population measure of attention predicts behavioral performance on individual trials

Visual attention improves perception for an attended location or feature and also modulates the responses of sensory neurons. In laboratory studies, the sensory stimuli and task instructions are held constant within an attentional condition, but despite experimenters' best efforts, attention likely varies from moment to moment. Because most previous studies have focused on single neurons, it has been impossible to use neuronal responses to identify attentional fluctuations and determine whether these are associated with changes in behavior. We show that an instantaneous measure of attention based on the responses of a modest number of neurons in area V4 of the rhesus monkey (Macaca mulatta) can reliably predict large changes in an animal's ability to perform a difficult psychophysical task. Unexpectedly, this measure shows that the amount of attention allocated at any moment to locations in opposite hemifields is uncorrelated, suggesting that animals allocate attention to each stimulus independently rather than moving their attentional focus from one location to another.

A method for investigating cortical control of stand and squat in conscious behavioral monkeys

Technical challenges to record cortical unit spikes in conscious behavior monkeys have been presenting constraints in the investigation of cortical control of lower limb. Here we report a novel experimental system for investigation of correlation between cortical neuronal signals and standing and squatting movement. The system consists of a specially designed chair, a standing and squatting task training paradigm and a neuron recording setup with microdrivable electrodes. With this system, a monkey can perform standing up and squatting down tasks while the head is relatively stationary to allow accurate single cortical unit recordings. The spike activities of single motor cortical units, the kinematics and muscle activities from monkeys functioning lower limb motion tasks were successfully recorded simultaneously for analyses of cortical control of lower limb functions and correlations between activity patterns of cortical neurons and lower limb muscles.