Preferential encoding of visual categories in parietal cortex compared with prefrontal cortex

Modular representations emerge in neural networks trained to perform context-dependent tasks

The brain has large-scale modular structure in the form of brain regions, which are thought to arise from constraints on connectivity and the physical geometry of the cortical sheet. In contrast, experimental and theoretical work has argued both for and against the existence of specialized sub-populations of neurons (modules) within single brain regions. By studying artificial neural networks, we show that this local modularity emerges to support context-dependent behavior, but only when the input is low-dimensional. No anatomical constraints are required. We also show when modular specialization emerges at the population level (different modules correspond to orthogonal subspaces). Modularity yields abstract representations, allows for rapid learning and generalization on novel tasks, and facilitates the rapid learning of related contexts. Non-modular representations facilitate the rapid learning of unrelated contexts. Our findings reconcile conflicting experimental results and make predictions for future experiments.

Primate superior colliculus is causally engaged in abstract higher-order cognition

The superior colliculus is an evolutionarily conserved midbrain region that is thought to mediate spatial orienting, including saccadic eye movements and covert spatial attention. Here, we reveal a role for the superior colliculus in higher-order cognition, independent of its role in spatial orienting. We trained rhesus macaques to perform an abstract visual categorization task that involved neither instructed eye movements nor differences in covert attention. We compared neural activity in the superior colliculus and the posterior parietal cortex, a region previously shown to causally contribute to abstract category decisions. The superior colliculus exhibits robust encoding of learned visual categories, which is stronger than in the posterior parietal cortex and arises at a similar latency in the two areas. Moreover, inactivation of the superior colliculus markedly impaired animals' category decisions. These results demonstrate that the primate superior colliculus mediates abstract, higher-order cognitive processes that have traditionally been attributed to the neocortex.

Choice-dependent delta-band neural trajectory during semantic category decision making in the human brain

Recent human brain imaging studies have identified widely distributed cortical areas that represent information about the meaning of language. Yet, the dynamic nature of widespread neural activity as a correlate of the semantic information processing remains poorly explored. Our state space analysis of electroencephalograms (EEGs) recorded during semantic match-to-category task show that depending on the semantic category and decision path chosen by participants, whole-brain delta-band dynamics follow distinct trajectories that are correlated with participants' response time on a trial-by-trial basis. Especially, the proximity of the neural trajectory to category decision-specific region in the state space was predictive of participants' decision-making reaction times. We also found that posterolateral regions primarily encoded word categories while postero-central regions encoded category decisions. Our results demonstrate the role of neural dynamics embedded in the evolving multivariate delta-band activity patterns in processing the semantic relatedness of words and the semantic category-based decision-making.

Task and stimulus coding in the multiple-demand network

In the human brain, a multiple-demand (MD) network plays a key role in cognitive control, with core components in lateral frontal, dorsomedial frontal and lateral parietal cortex, and multivariate activity patterns that discriminate the contents of many cognitive activities. In prefrontal cortex of the behaving monkey, different cognitive operations are associated with very different patterns of neural activity, while details of a particular stimulus are encoded as small variations on these basic patterns (Sigala et al, 2008). Here, using the advanced fMRI methods of the Human Connectome Project and their 360-region cortical parcellation, we searched for a similar result in MD activation patterns. In each parcel, we compared multivertex patterns for every combination of three tasks (working memory, task-switching, and stop-signal) and two stimulus classes (faces and buildings). Though both task and stimulus category were discriminated in every cortical parcel, the strength of discrimination varied strongly across parcels. The different cognitive operations of the three tasks were strongly discriminated in MD regions. Stimulus categories, in contrast, were most strongly discriminated in a large region of primary and higher visual cortex, and intriguingly, in both parietal and frontal lobe regions adjacent to core MD regions. In the monkey, frontal neurons show a strong pattern of nonlinear mixed selectivity, with activity reflecting specific conjunctions of task events. In our data, however, there was limited evidence for mixed selectivity; throughout the brain, discriminations of task and stimulus combined largely linearly, with a small nonlinear component. In MD regions, human fMRI data recapitulate some but not all aspects of electrophysiological data from nonhuman primates.

Category representation in primary visual cortex after visual perceptual learning

The visual perceptual learning (VPL) leads to long-term enhancement of visual task performance. The subjects are often trained to link different visual stimuli to several options, such as the widely used two-alternative forced choice (2AFC) task, which involves an implicit categorical decision. The enhancement of performance has been related to the specific changes of neural activities, but few studies investigate the effects of categorical responding on the changes of neural activities. Here we investigated whether the neural activities would exhibit the categorical characteristics if the subjects are requested to respond visual stimuli in a categorical manner during VPL. We analyzed the neural activities of two monkeys in a contour detection VPL. We found that the neural activities in primary visual cortex (V1) converge to one pattern if the contour can be detected by monkey and another pattern if the contour cannot be detected, exhibiting a kind of category learning that the neural representations of detectable contour become less selective for number of bars forming contour and diverge from the representations of undetectable contour.

Supplementary Information

The online version contains supplementary material available at 10.1007/s11571-022-09926-8.

Dynamics and maintenance of categorical responses in primary auditory cortex during task engagement

Grouping sets of sounds into relevant categories is an important cognitive ability that enables the association of stimuli with appropriate goal-directed behavioral responses. In perceptual tasks, the primary auditory cortex (A1) assumes a prominent role by concurrently encoding both sound sensory features and task-related variables. Here, we sought to explore the role of A1 in the initiation of sound categorization, shedding light on its involvement in this cognitive process. We trained ferrets to discriminate click trains of different rates in a Go/No-Go delayed categorization task and recorded neural activity during both active behavior and passive exposure to the same sounds. Purely categorical response components were extracted and analyzed separately from sensory responses to reveal their contributions to the overall population response throughout the trials. We found that categorical activity emerged during sound presentation in the population average and was present in both active behavioral and passive states. However, upon task engagement, categorical responses to the No-Go category became suppressed in the population code, leading to an asymmetrical representation of the Go stimuli relative to the No-Go sounds and pre-stimulus baseline. The population code underwent an abrupt change at stimulus offset, with sustained responses after the Go sounds during the delay period. Notably, the categorical responses observed during the stimulus period exhibited a significant correlation with those extracted from the delay epoch, suggesting an early involvement of A1 in stimulus categorization.

Parietal-driven visual working memory representation in occipito-temporal cortex

Human fMRI studies have documented extensively that the content of visual working memory (VWM) can be reliably decoded from fMRI voxel response patterns during the delay period in both the occipito-temporal cortex (OTC), including early visual areas (EVC), and the posterior parietal cortex (PPC).1,2,3,4 Further work has revealed that VWM signal in OTC is largely sustained by feedback from associative areas such as prefrontal cortex (PFC) and PPC.4,5,6,7,8,9 It is unclear, however, if feedback during VWM simply restores sensory representations initially formed in OTC or if it can reshape the representational content of OTC during VWM delay. Taking advantage of a recent finding showing that object representational geometry differs between OTC and PPC in perception,10 here we find that, during VWM delay, the object representational geometry in OTC becomes more aligned with that of PPC during perception than with itself during perception. This finding supports the role of feedback in shaping the content of VWM in OTC, with the VWM content of OTC more determined by information retained in PPC than by the sensory information initially encoded in OTC.

A source for category-induced global effects of feature-based attention in human prefrontal cortex

Global effects of feature-based attention (FBA) are generally limited to stimuli sharing the same or similar features, as hypothesized in the "feature-similarity gain model." Visual perception, however, often reflects categories acquired via experience/learning; whether the global-FBA effect can be induced by the categorized features remains unclear. Here, human subjects were trained to classify motion directions into two discrete categories and perform a classical motion-based attention task. We found a category-induced global-FBA effect in both the middle temporal area (MT+) and frontoparietal areas, where attention to a motion direction globally spread to unattended motion directions within the same category, but not to those in a different category. Effective connectivity analysis showed that the category-induced global-FBA effect in MT+ was derived by feedback from the inferior frontal junction (IFJ). Altogether, our study reveals a category-induced global-FBA effect and identifies a source for this effect in human prefrontal cortex, implying that FBA is of greater ecological significance than previously thought.

Posterior Parietal Cortex Plays a Causal Role in Abstract Memory-Based Visual Categorical Decisions

Neural activity in the lateral intraparietal cortex (LIP) correlates with both sensory evaluation and motor planning underlying visuomotor decisions. We previously showed that LIP plays a causal role in visually-based perceptual and categorical decisions, and preferentially contributes to evaluating sensory stimuli over motor planning. In that study, however, monkeys reported their decisions with a saccade to a colored target associated with the correct motion category or direction. Since LIP is known to play a role in saccade planning, it remains unclear whether LIP's causal role in such decisions extend to decision-making tasks which do not involve saccades. Here, we employed reversible pharmacological inactivation of LIP neural activity while two male monkeys performed delayed match to category (DMC) and delayed match to sample (DMS) tasks. In both tasks, monkeys needed to maintain gaze fixation throughout the trial and report whether a test stimulus was a categorical match or nonmatch to the previous sample stimulus by releasing a touch bar. LIP inactivation impaired monkeys' behavioral performance in both tasks, with deficits in both accuracy and reaction time (RT). Furthermore, we recorded LIP neural activity in the DMC task targeting the same cortical locations as in the inactivation experiments. We found significant neural encoding of the sample category, which was correlated with monkeys' categorical decisions in the DMC task. Taken together, our results demonstrate that LIP plays a generalized role in visual categorical decisions independent of the task-structure and motor response modality.SIGNIFICANCE STATEMENT Neural activity in the lateral intraparietal cortex (LIP) correlates with perceptual and categorical decisions, in addition to its role in mediating saccadic eye movements. Past work found that LIP is causally involved in visual decisions that are rapidly reported by saccades in a reaction time based decision making task. Here we use reversible inactivation of LIP to test whether LIP is also causally involved in visual decisions when reported by hand movements during delayed matching tasks. Here we show that LIP inactivation impaired monkeys' task performance during both memory-based discrimination and categorization tasks. These results demonstrate that LIP plays a generalized role in visual categorical decisions independent of the task-structure and motor response modality.

Deciphering the clinico-radiological heterogeneity of dysexecutive Alzheimer's disease

Dysexecutive Alzheimer's disease (dAD) manifests as a progressive dysexecutive syndrome without prominent behavioral features, and previous studies suggest clinico-radiological heterogeneity within this syndrome. We uncovered this heterogeneity using unsupervised machine learning in 52 dAD patients with multimodal imaging and cognitive data. A spectral decomposition of covariance between FDG-PET images yielded six latent factors ("eigenbrains") accounting for 48% of variance in patterns of hypometabolism. These eigenbrains differentially related to age at onset, clinical severity, and cognitive performance. A hierarchical clustering on the eigenvalues of these eigenbrains yielded four dAD subtypes, i.e. "left-dominant," "right-dominant," "bi-parietal-dominant," and "heteromodal-diffuse." Patterns of FDG-PET hypometabolism overlapped with those of tau-PET distribution and MRI neurodegeneration for each subtype, whereas patterns of amyloid deposition were similar across subtypes. Subtypes differed in age at onset and clinical severity where the heteromodal-diffuse exhibited a worse clinical picture, and the bi-parietal had a milder clinical presentation. We propose a conceptual framework of executive components based on the clinico-radiological associations observed in dAD. We demonstrate that patients with dAD, despite sharing core clinical features, are diagnosed with variability in their clinical and neuroimaging profiles. Our findings support the use of data-driven approaches to delineate brain-behavior relationships relevant to clinical practice and disease physiology.

Hierarchical categorization learning is associated with representational changes in the dorsal striatum and posterior frontal and parietal cortex

Learning and recognition can be improved by sorting novel items into categories and subcategories. Such hierarchical categorization is easy when it can be performed according to learned rules (e.g., "if car, then automatic or stick shift" or "if boat, then motor or sail"). Here, we present results showing that human participants acquire categorization rules for new visual hierarchies rapidly, and that, as they do, corresponding hierarchical representations of the categorized stimuli emerge in patterns of neural activation in the dorsal striatum and in posterior frontal and parietal cortex. Participants learned to categorize novel visual objects into a hierarchy with superordinate and subordinate levels based on the objects' shape features, without having been told the categorization rules for doing so. On each trial, participants were asked to report the category and subcategory of the object, after which they received feedback about the correctness of their categorization responses. Participants trained over the course of a one-hour-long session while their brain activation was measured using functional magnetic resonance imaging. Over the course of training, significant hierarchy learning took place as participants discovered the nested categorization rules, as evidenced by the occurrence of a learning trial, after which performance suddenly increased. This learning was associated with increased representational strength of the newly acquired hierarchical rules in a corticostriatal network including the posterior frontal and parietal cortex and the dorsal striatum. We also found evidence suggesting that reinforcement learning in the dorsal striatum contributed to hierarchical rule learning.

Shared Neural Activity But Distinct Neural Dynamics for Cognitive Control in Monkey Prefrontal and Parietal Cortex

To better understand how prefrontal networks mediate forms of cognitive control disrupted in schizophrenia, we translated a variant of the AX continuous performance task that measures specific deficits in the human disease to 2 male monkeys and recorded neurons in PFC and parietal cortex during task performance. In the task, contextual information instructed by cue stimuli determines the response required to a subsequent probe stimulus. We found parietal neurons encoding the behavioral context instructed by cues that exhibited nearly identical activity to their prefrontal counterparts (Blackman et al., 2016). This neural population switched their preference for stimuli over the course of the trial depending on whether the stimuli signaled the need to engage cognitive control to override a prepotent response. Cues evoked visual responses that appeared in parietal neurons first, whereas population activity encoding contextual information instructed by cues was stronger and more persistent in PFC. Increasing cognitive control demand biased the representation of contextual information toward the PFC and augmented the temporal correlation of task-defined information encoded by neurons in the two areas. Oscillatory dynamics in local field potentials differed between cortical areas and carried as much information about task conditions as spike rates. We found that, at the single-neuron level, patterns of activity evoked by the task were nearly identical between the two cortical areas. Nonetheless, distinct population dynamics in PFC and parietal cortex were evident. suggesting differential contributions to cognitive control.SIGNIFICANCE STATEMENT We recorded neural activity in PFC and parietal cortex of monkeys performing a task that measures cognitive control deficits in schizophrenia. This allowed us to characterize computations performed by neurons in the two areas to support forms of cognitive control disrupted in the disease. Subpopulations of neurons in the two areas exhibited parallel modulations in firing rate; and as a result, all patterns of task-evoked activity were distributed between PFC and parietal cortex. This included the presence in both cortical areas of neurons reflecting proactive and reactive cognitive control dissociated from stimuli or responses in the task. However, differences in the timing, strength, synchrony, and correlation of information encoded by neural activity were evident, indicating differential contributions to cognitive control.

Low-dimensional encoding of decisions in parietal cortex reflects long-term training history

Neurons in parietal cortex exhibit task-related activity during decision-making tasks. However, it remains unclear how long-term training to perform different tasks over months or even years shapes neural computations and representations. We examine lateral intraparietal area (LIP) responses during a visual motion delayed-match-to-category task. We consider two pairs of male macaque monkeys with different training histories: one trained only on the categorization task, and another first trained to perform fine motion-direction discrimination (i.e., pretrained). We introduce a novel analytical approach-generalized multilinear models-to quantify low-dimensional, task-relevant components in population activity. During the categorization task, we found stronger cosine-like motion-direction tuning in the pretrained monkeys than in the category-only monkeys, and that the pretrained monkeys' performance depended more heavily on fine discrimination between sample and test stimuli. These results suggest that sensory representations in LIP depend on the sequence of tasks that the animals have learned, underscoring the importance of considering training history in studies with complex behavioral tasks.

Abstract representations emerge naturally in neural networks trained to perform multiple tasks

Humans and other animals demonstrate a remarkable ability to generalize knowledge across distinct contexts and objects during natural behavior. We posit that this ability to generalize arises from a specific representational geometry, that we call abstract and that is referred to as disentangled in machine learning. These abstract representations have been observed in recent neurophysiological studies. However, it is unknown how they emerge. Here, using feedforward neural networks, we demonstrate that the learning of multiple tasks causes abstract representations to emerge, using both supervised and reinforcement learning. We show that these abstract representations enable few-sample learning and reliable generalization on novel tasks. We conclude that abstract representations of sensory and cognitive variables may emerge from the multiple behaviors that animals exhibit in the natural world, and, as a consequence, could be pervasive in high-level brain regions. We also make several specific predictions about which variables will be represented abstractly.

Primate superior colliculus is engaged in abstract higher-order cognition

Categorization is a fundamental cognitive process by which the brain assigns stimuli to behaviorally meaningful groups. Investigations of visual categorization in primates have identified a hierarchy of cortical areas that are involved in the transformation of sensory information into abstract category representations. However, categorization behaviors are ubiquitous across diverse animal species, even those without a neocortex, motivating the possibility that subcortical regions may contribute to abstract cognition in primates. One candidate structure is the superior colliculus (SC), an evolutionarily conserved midbrain region that, although traditionally thought to mediate only reflexive spatial orienting, is involved in cognitive tasks that require spatial orienting. Here, we reveal a novel role of the primate SC in abstract, higher-order visual cognition. We compared neural activity in the SC and the posterior parietal cortex (PPC), a region previously shown to causally contribute to category decisions, while monkeys performed a visual categorization task in which they report their decisions with a hand movement. The SC exhibits stronger and shorter-latency category encoding than the PPC, and inactivation of the SC markedly impairs monkeys' category decisions. These results extend SC's established role in spatial orienting to abstract, non-spatial cognition.

Using Nonhuman Primate Models to Reverse-Engineer Prefrontal Circuit Failure Underlying Cognitive Deficits in Schizophrenia

In this chapter, I review studies in nonhuman primates that emulate the circuit failure in prefrontal cortex responsible for working memory and cognitive control deficits in schizophrenia. These studies have characterized how synaptic malfunction, typically induced by blockade of NMDAR, disrupts neural function and computation in prefrontal networks to explain errors in cognitive tasks that are seen in schizophrenia. This work is finding causal relationships between pathogenic events of relevance to schizophrenia at vastly different levels of scale, from synapses, to neurons, local, circuits, distributed networks, computation, and behavior. Pharmacological manipulation, the dominant approach in primate models, has limited construct validity for schizophrenia pathogenesis, as the disease results from a complex interplay between environmental, developmental, and genetic factors. Genetic manipulation replicating schizophrenia risk is more advanced in rodent models. Nonetheless, gene manipulation in nonhuman primates is rapidly advancing, and primate developmental models have been established. Integration of large scale neural recording, genetic manipulation, and computational modeling in nonhuman primates holds considerable potential to provide a crucial schizophrenia model moving forward. Data generated by this approach is likely to fill several crucial gaps in our understanding of the causal sequence leading to schizophrenia in humans. This causal chain presents a vexing problem largely because it requires understanding how events at very different levels of scale relate to one another, from genes to circuits to cognition to social interactions. Nonhuman primate models excel here. They optimally enable discovery of causal relationships across levels of scale in the brain that are relevant to cognitive deficits in schizophrenia. The mechanistic understanding of prefrontal circuit failure they promise to provide may point the way to more effective therapeutic interventions to restore function to prefrontal networks in the disease.

Abstract Encoding of Categorical Decisions in Medial Superior Temporal and Lateral Intraparietal Cortices

Categorization is an essential cognitive and perceptual process for decision-making and recognition. The posterior parietal cortex, particularly the lateral intraparietal (LIP) area has been suggested to transform visual feature encoding into abstract categorical representations. By contrast, areas closer to sensory input, such as the middle temporal (MT) area, encode stimulus features but not more abstract categorical information during categorization tasks. Here, we compare the contributions of the medial superior temporal (MST) and LIP areas in category computation by recording neuronal activity in both areas from two male rhesus macaques trained to perform a visual motion categorization task. MST is a core motion-processing region interconnected with MT and is often considered an intermediate processing stage between MT and LIP. We show that MST exhibits robust decision-correlated motion category encoding and working memory encoding similar to LIP, suggesting that MST plays a substantial role in cognitive computation, extending beyond its widely recognized role in visual motion processing.SIGNIFICANCE STATEMENT Categorization requires assigning incoming sensory stimuli into behaviorally relevant groups. Previous work found that parietal area LIP shows a strong encoding of the learned category membership of visual motion stimuli, while visual area MT shows strong direction tuning but not category tuning during a motion direction categorization task. Here we show that the medial superior temporal (MST) area, a visual motion-processing region interconnected with both LIP and MT, shows strong visual category encoding similar to that observed in LIP. This suggests that MST plays a greater role in abstract cognitive functions, extending beyond its well known role in visual motion processing.

Coding of latent variables in sensory, parietal, and frontal cortices during closed-loop virtual navigation

We do not understand how neural nodes operate and coordinate within the recurrent action-perception loops that characterize naturalistic self-environment interactions. Here, we record single-unit spiking activity and local field potentials (LFPs) simultaneously from the dorsomedial superior temporal area (MSTd), parietal area 7a, and dorsolateral prefrontal cortex (dlPFC) as monkeys navigate in virtual reality to 'catch fireflies'. This task requires animals to actively sample from a closed-loop virtual environment while concurrently computing continuous latent variables: (i) the distance and angle travelled (i.e., path integration) and (ii) the distance and angle to a memorized firefly location (i.e., a hidden spatial goal). We observed a patterned mixed selectivity, with the prefrontal cortex most prominently coding for latent variables, parietal cortex coding for sensorimotor variables, and MSTd most often coding for eye movements. However, even the traditionally considered sensory area (i.e., MSTd) tracked latent variables, demonstrating path integration and vector coding of hidden spatial goals. Further, global encoding profiles and unit-to-unit coupling (i.e., noise correlations) suggested a functional subnetwork composed by MSTd and dlPFC, and not between these and 7a, as anatomy would suggest. We show that the greater the unit-to-unit coupling between MSTd and dlPFC, the more the animals' gaze position was indicative of the ongoing location of the hidden spatial goal. We suggest this MSTd-dlPFC subnetwork reflects the monkeys' natural and adaptive task strategy wherein they continuously gaze toward the location of the (invisible) target. Together, these results highlight the distributed nature of neural coding during closed action-perception loops and suggest that fine-grain functional subnetworks may be dynamically established to subserve (embodied) task strategies.

More Prominent Nonlinear Mixed Selectivity in the Dorsolateral Prefrontal than Posterior Parietal Cortex

Neurons in the dorsolateral prefrontal cortex (dlPFC) and posterior parietal cortex (PPC) are activated by different cognitive tasks and respond differently to the same stimuli depending on task. The conjunctive representations of multiple tasks in nonlinear fashion in single neuron activity, is known as nonlinear mixed selectivity (NMS). Here, we compared NMS in a working memory task in areas 8a and 46 of the dlPFC and 7a and lateral intraparietal cortex (LIP) of the PPC in macaque monkeys. NMS neurons were more frequent in dlPFC than in PPC and this was attributed to more cells gaining selectivity in the course of a trial. Additionally, in our task, the subjects' behavioral performance improved within a behavioral session as they learned the session-specific statistics of the task. The magnitude of NMS in the dlPFC also increased as a function of time within a single session. On the other hand, we observed minimal rotation of population responses and no appreciable differences in NMS between correct and error trials in either area. Our results provide direct evidence demonstrating a specialization in NMS between dlPFC and PPC and reveal mechanisms of neural selectivity in areas recruited in working memory tasks.

Working Memory: From Neural Activity to the Sentient Mind

Working memory (WM) is the ability to maintain and manipulate information in the conscious mind over a timescale of seconds. This ability is thought to be maintained through the persistent discharges of neurons in a network of brain areas centered on the prefrontal cortex, as evidenced by neurophysiological recordings in nonhuman primates, though both the localization and the neural basis of WM has been a matter of debate in recent years. Neural correlates of WM are evident in species other than primates, including rodents and corvids. A specialized network of excitatory and inhibitory neurons, aided by neuromodulatory influences of dopamine, is critical for the maintenance of neuronal activity. Limitations in WM capacity and duration, as well as its enhancement during development, can be attributed to properties of neural activity and circuits. Changes in these factors can be observed through training-induced improvements and in pathological impairments. WM thus provides a prototypical cognitive function whose properties can be tied to the spiking activity of brain neurons. © 2021 American Physiological Society. Compr Physiol 11:1-41, 2021.

Distributed functions of prefrontal and parietal cortices during sequential categorical decisions

Comparing sequential stimuli is crucial for guiding complex behaviors. To understand mechanisms underlying sequential decisions, we compared neuronal responses in the prefrontal cortex (PFC), the lateral intraparietal (LIP), and medial intraparietal (MIP) areas in monkeys trained to decide whether sequentially presented stimuli were from matching (M) or nonmatching (NM) categories. We found that PFC leads M/NM decisions, whereas LIP and MIP appear more involved in stimulus evaluation and motor planning, respectively. Compared to LIP, PFC showed greater nonlinear integration of currently visible and remembered stimuli, which correlated with the monkeys' M/NM decisions. Furthermore, multi-module recurrent networks trained on the same task exhibited key features of PFC and LIP encoding, including nonlinear integration in the PFC-like module, which was causally involved in the networks' decisions. Network analysis found that nonlinear units have stronger and more widespread connections with input, output, and within-area units, indicating putative circuit-level mechanisms for sequential decisions.

Computation Through Neural Population Dynamics

Significant experimental, computational, and theoretical work has identified rich structure within the coordinated activity of interconnected neural populations. An emerging challenge now is to uncover the nature of the associated computations, how they are implemented, and what role they play in driving behavior. We term this computation through neural population dynamics. If successful, this framework will reveal general motifs of neural population activity and quantitatively describe how neural population dynamics implement computations necessary for driving goal-directed behavior. Here, we start with a mathematical primer on dynamical systems theory and analytical tools necessary to apply this perspective to experimental data. Next, we highlight some recent discoveries resulting from successful application of dynamical systems. We focus on studies spanning motor control, timing, decision-making, and working memory. Finally, we briefly discuss promising recent lines of investigation and future directions for the computation through neural population dynamics framework.

Linking Neuronal Direction Selectivity to Perceptual Decisions About Visual Motion

Psychophysical and neurophysiological studies of responses to visual motion have converged on a consistent set of general principles that characterize visual processing of motion information. Both types of approaches have shown that the direction and speed of target motion are among the most important encoded stimulus properties, revealing many parallels between psychophysical and physiological responses to motion. Motivated by these parallels, this review focuses largely on more direct links between the key feature of the neuronal response to motion, direction selectivity, and its utilization in memory-guided perceptual decisions. These links were established during neuronal recordings in monkeys performing direction discriminations, but also by examining perceptual effects of widespread elimination of cortical direction selectivity produced by motion deprivation during development. Other approaches, such as microstimulation and lesions, have documented the importance of direction-selective activity in the areas that are active during memory-guided direction comparisons, area MT and the prefrontal cortex, revealing their likely interactions during behavioral tasks.

Neural Hierarchy of Color Categorization: From Prototype Encoding to Boundary Encoding

A long-standing debate exists on how our brain assigns the fine-grained perceptual representation of color into discrete color categories. Recent functional magnetic resonance imaging (fMRI) studies have identified several regions as the candidate loci of color categorization, including the visual cortex, language-related areas, and non-language-related frontal regions, but the evidence is mixed. Distinct from most studies that emphasized the representational differences between color categories, the current study focused on the variability among members within a category (e.g., category prototypes and boundaries) to reveal category encoding in the brain. We compared and modeled brain activities evoked by color stimuli with varying distances from the category boundary in an active categorization task. The frontal areas, including the inferior and middle frontal gyri, medial superior frontal cortices, and insular cortices, showed larger responses for colors near the category boundary than those far from the boundary. In addition, the visual cortex encodes both within-category variability and cross-category differences. The left V1 in the calcarine showed greater responses to colors at the category center than to those far from the boundary, and the bilateral V4 showed enhanced responses for colors at the category center as well as colors around the boundary. The additional representational similarity analyses (RSA) revealed that the bilateral insulae and V4a carried information about cross-category differences, as cross-category colors exhibited larger dissimilarities in brain patterns than within-category colors. Our study suggested a hierarchically organized network in the human brain during active color categorization, with frontal (both lateral and medial) areas supporting domain-general decisional processes and the visual cortex encoding category structure and differences, likely due to top-down modulation.

Towards a better understanding of information storage in visual working memory

Chota and Van der Stigchel (this issue), Iamshchinina, Christophel, Gayet, and Rademaker (this issue), Lorenc and Sreenivasa (this issue), and Teng and Postle (this issue) each present a commentary regarding Xu (2020) where I conclude that sensory regions are nonessential for the storage of information in visual working memory (VWM). They argue instead that sensory regions are critical to VWM storage. Here I briefly reiterate some of the key evidence against this account, some of which has not been accounted by the four commentaries. I also provide a detailed reanalysis of why the main evidence supporting this account may be problematic. Collectively, existence evidence from human neuroimaging and TMS studies and that from monkey neurophysiology studies does not provide strong support for the sensory storage account of VWM. To form an accurate understanding of the distinctive role each brain region may play in perception and VWM as well as how they may interact to collectively support a VWM task, it is important that we properly survey and evaluate all the available evidence.

Category learning in a transitive inference paradigm

The implied order of a ranked set of visual images can be learned without reliance on information that explicitly signals their order. Such learning is difficult to explain by associative mechanisms, but can be accounted for by cognitive representations and processes such as transitive inference. Our study sought to determine if those processes also apply to learning categories of images. We asked whether participants can (a) infer that stimulus images belonged to familiar categories, even when the images for each trial were unique, and (b) sort those categories into an ordering that obeys transitivity. Participants received minimal verbal instruction and a single session of training. Despite this, they learned the implied order of lists of fixed stimuli and lists of ordered categories, using trial-unique exemplars. We trained two groups, one for which stimuli were constant throughout training and testing (n = 60), and one for which exemplars of each category were trial-unique (n = 50). Our findings suggest that differing cognitive processes may underpin serial learning when learning about specific stimuli as opposed to stimulus categories.

The functional roles of neural remapping in cortex

Remapping is a property of some cortical and subcortical neurons that update their responses around the time of an eye movement to account for the shift of stimuli on the retina due to the saccade. Physiologically, remapping is traditionally tested by briefly presenting a single stimulus around the time of the saccade and looking at the onset of the response and the locations in space to which the neuron is responsive. Here we suggest that a better way to understand the functional role of remapping is to look at the time at which the neural signal emerges when saccades are made across a stable scene. Based on data obtained using this approach, we suggest that remapping in the lateral intraparietal area is sufficient to play a role in maintaining visual stability across saccades, whereas in the frontal eye field, remapped activity carries information that affects future saccadic choices and, in a separate subset of neurons, is used to maintain a map of locations in the scene that have been previously fixated.

The roles of the lateral intraparietal area and frontal eye field in guiding eye movements in free viewing search behavior

The lateral intraparietal area (LIP) and frontal eye field (FEF) have been shown to play significant roles in oculomotor control, yet most studies have found that the two areas behave similarly. To identify the unique roles each area plays in guiding eye movements, we recorded 200 LIP neurons and 231 FEF neurons from four animals performing a free viewing visual foraging task. We analyzed how neuronal responses were modulated by stimulus identity and the animals' choice of where to make a saccade. We additionally analyzed the comodulation of the sensory signals and the choice signal to identify how the sensory signals drove the choice. We found a clearly defined division of labor: LIP provided a stable map integrating task rules and stimulus identity, whereas FEF responses were dynamic, representing more complex information and, just before the saccade, were integrated with task rules and stimulus identity to decide where to move the eye.NEW & NOTEWORTHY The lateral intrapareital area (LIP) and frontal eye field (FEF) are known to contribute to guiding eye movements, but little is known about the unique roles that each area plays. Using a free viewing visual search task, we found that LIP provides a stable map of the visual world, integrating task rules and stimulus identity. FEF activity is consistently modulated by more complex information but, just before the saccade, integrates all the information to make the final decision about where to move.

Drifts in Prefrontal and Parietal Neuronal Activity Influence Working Memory Judgments

The dorsolateral prefrontal cortex (dlPFC) plays a critical role in spatial working memory and its activity predicts behavioral responses in delayed response tasks. Here, we addressed if this predictive ability extends to other working memory tasks and if it is present in other brain areas. We trained monkeys to remember the location of a stimulus and determine whether a second stimulus appeared at the same location or not. Neurophysiological recordings were performed in the dorsolateral prefrontal cortex and posterior parietal cortex (PPC). We hypothesized that random drifts causing the peak activity of the network to move away from the first stimulus location and toward the location of the second stimulus would result in categorical errors. Indeed, for both areas, in nonmatching trials, when the first stimulus appeared in a neuron's preferred location, the neuron showed significantly higher firing rates in correct than in error trials; and vice versa, when the first stimulus appeared at a nonpreferred location, activity in error trials was higher than in correct. The results indicate that the activity of both dlPFC and PPC neurons is predictive of categorical judgments of information maintained in working memory, and neuronal firing rate deviations are revealing of the contents of working memory.

Evidence accumulation under uncertainty - a neural marker of emerging choice and urgency

To interact meaningfully with its environment, an agent must integrate external information with its own internal states. However, information about the environment is often noisy. In this study, we identify a neural correlate that tracks how asymmetries between competing alternatives evolve over the course of a decision. In our task participants had to monitor a stream of discrete visual stimuli over time and decide whether or not to act, on the basis of either strong or ambiguous evidence. We found that the classic P3 event-related potential evoked by sequential evidence items tracked decision-making processes and predicted participants' categorical choices on a single trial level, both when evidence was strong and when it was ambiguous. The P3 amplitudes in response to evidence supporting the eventually selected option increased over trial time as decisions evolved, being maximally different from the P3 amplitudes evoked by competing evidence at the time of decision. Computational modelling showed that both the neural dynamics and behavioural primacy and recency effects can be explained by a combination of (a) competition between mutually inhibiting accumulators for the two categorical choice outcomes, and (b) a context-dependant urgency signal. In conditions where evidence was presented at a low rate, urgency increased faster than in conditions when evidence was very frequent. We also found that the readiness potential, a classic marker of endogenously initiated actions, was observed preceding movements in all conditions - even when those were strongly driven by external evidence.

Reward-related choices determine information timing and flow across macaque lateral prefrontal cortex

Prefrontal cortex is critical for cognition. Although much is known about the representation of cognitive variables in the prefrontal cortex, much less is known about the spatio-temporal neural dynamics that underlie cognitive operations. In the present study, we examined information timing and flow across the lateral prefrontal cortex (LPFC), while monkeys carried out a two-armed bandit reinforcement learning task in which they had to learn to select rewarding actions or rewarding objects. When we analyzed signals independently within subregions of the LPFC, we found a task-specific, caudo-rostral gradient in the strength and timing of signals related to chosen objects and chosen actions. In addition, when we characterized information flow among subregions, we found that information flow from action to object representations was stronger from the dorsal to ventral LPFC, and information flow from object to action representations was stronger from the ventral to dorsal LPFC. The object to action effects were more pronounced in object blocks, and also reflected learning specifically in these blocks. These results suggest anatomical segregation followed by the rapid integration of information within the LPFC.

Previous studies provided conflicting evidence on the functional organization of the lateral prefrontal cortex. The authors show task-specific information flows along the caudo-rostral and dorso-ventral axes, reflecting the cognitive process of identifying the location or identity of a valuable object.

Mouse visual cortex areas represent perceptual and semantic features of learned visual categories

Associative memories are stored in distributed networks extending across multiple brain regions. However, it is unclear to what extent sensory cortical areas are part of these networks. Using a paradigm for visual category learning in mice, we investigated whether perceptual and semantic features of learned category associations are already represented at the first stages of visual information processing in the neocortex. Mice learned categorizing visual stimuli, discriminating between categories and generalizing within categories. Inactivation experiments showed that categorization performance was contingent on neuronal activity in the visual cortex. Long-term calcium imaging in nine areas of the visual cortex identified changes in feature tuning and category tuning that occurred during this learning process, most prominently in the postrhinal area (POR). These results provide evidence for the view that associative memories form a brain-wide distributed network, with learning in early stages shaping perceptual representations and supporting semantic content downstream.

Visual number sense in untrained deep neural networks

Number sense, the ability to estimate numerosity, is observed in naïve animals, but how this cognitive function emerges in the brain remains unclear. Here, using an artificial deep neural network that models the ventral visual stream of the brain, we show that number-selective neurons can arise spontaneously, even in the complete absence of learning. We also show that the responses of these neurons can induce the abstract number sense, the ability to discriminate numerosity independent of low-level visual cues. We found number tuning in a randomly initialized network originating from a combination of monotonically decreasing and increasing neuronal activities, which emerges spontaneously from the statistical properties of bottom-up projections. We confirmed that the responses of these number-selective neurons show the single- and multineuron characteristics observed in the brain and enable the network to perform number comparison tasks. These findings provide insight into the origin of innate cognitive functions.

Interaction between neuronal encoding and population dynamics during categorization task switching in parietal cortex

Primates excel at categorization, a cognitive process for assigning stimuli into behaviorally relevant groups. Categories are encoded in multiple brain areas and tasks, yet it remains unclear how neural encoding and dynamics support cognitive tasks with different demands. We recorded from parietal cortex during flexible switching between categorization tasks with distinct cognitive and motor demands and also studied recurrent neural networks (RNNs) trained on the same tasks. In the one-interval categorization task (OIC), monkeys rapidly reported their decisions with a saccade. In the delayed match-to-category (DMC) task, monkeys decided whether sequentially presented stimuli were categorical matches. Neuronal category encoding generalized across tasks, but categorical encoding was more binary-like in the DMC task and more graded in the OIC task. Furthermore, analysis of trained RNNs supports the hypothesis that binary-like encoding in DMC arises through compression of graded feature encoding by attractor dynamics underlying stimulus maintenance and/or comparison in working memory.

Response modality-dependent categorical choice representations for vibrotactile comparisons

Previous electrophysiological studies in monkeys and humans suggest that premotor regions are the primary loci for the encoding of perceptual choices during vibrotactile comparisons. However, these studies employed paradigms wherein choices were inextricably linked with the stimulus order and selection of manual movements. It remains largely unknown how vibrotactile choices are represented when they are decoupled from these sensorimotor components of the task. To address this question, we used fMRI-MVPA and a variant of the vibrotactile frequency discrimination task which enabled the isolation of choice-related signals from those related to stimulus order and selection of the manual decision reports. We identified the left contralateral dorsal premotor cortex (PMd) and intraparietal sulcus (IPS) as carrying information about vibrotactile choices. Our finding provides empirical evidence for an involvement of the PMd and IPS in vibrotactile decisions that goes above and beyond the coding of stimulus order and specific action selection. Considering findings from recent studies in animals, we speculate that the premotor region likely serves as a temporary storage site for information necessary for the specification of concrete manual movements, while the IPS might be more directly involved in the computation of choice. Moreover, this finding replicates results from our previous work using an oculomotor variant of the task, with the important difference that the informative premotor cluster identified in the previous work was centered in the bilateral frontal eye fields rather than in the PMd. Evidence from these two studies indicates that categorical choices in human vibrotactile comparisons are represented in a response modality-dependent manner.

Biased Neural Representation of Feature-Based Attention in the Human Frontoparietal Network

Selective attention is a core cognitive function for efficient processing of information. Although it is well known that attention can modulate neural responses in many brain areas, the computational principles underlying attentional modulation remain unclear. Contrary to the prevailing view of a high-dimensional, distributed neural representation, here we show a surprisingly simple, biased neural representation for feature-based attention in a large dataset including five human fMRI studies. We found that when human participants (both sexes) selected one feature from a compound stimulus, voxels in many cortical areas responded consistently higher to one attended feature over the other. This univariate bias was consistent across brain areas within individual subjects. Importantly, this univariate bias showed a progressively stronger magnitude along the cortical hierarchy. In frontoparietal areas, the bias was strongest and contributed largely to pattern-based decoding, whereas early visual areas lacked such a bias. These findings suggest a gradual transition from a more analog to a more abstract representation of attentional priority along the cortical hierarchy. Biased neural responses in high-level areas likely reflect a low-dimensional neural code that can facilitate a robust representation and simple readout of cognitive variables.SIGNIFICANCE STATEMENT It is typically assumed that cognitive variables are represented by distributed population activities. Although this view is rooted in decades of work in the sensory system, it has not been rigorously tested at different levels of cortical hierarchy. Here we show a novel, low-dimensional coding scheme that dominated the representation of feature-based attention in frontoparietal areas. The simplicity of such a biased code may confer a robust representation of cognitive variables, such as attentional selection, working memory, and decision-making.

Neuronal Correlates of Many-To-One Sensorimotor Mapping in Lateral Intraparietal Cortex

Efficiently mapping sensory stimuli onto motor programs is crucial for rapidly choosing appropriate behavioral responses. While neuronal mechanisms underlying simple, one-to-one sensorimotor mapping have been extensively studied, how the brain achieves complex, many-to-one sensorimotor mapping remains unclear. Here, we recorded single neuron activity from the lateral intraparietal (LIP) cortex of monkeys trained to map multiple spatial positions of visual cue onto two opposite saccades. We found that LIP neurons' activity was consistent with directly mapping multiple cue positions to the associated saccadic direction (SDir) regardless of whether the visual cue appeared in or outside neurons' receptive fields. Unlike the explicit encoding of the visual categories, such cue-target mapping (CTM)-related activity covaried with the associated SDirs. Furthermore, the CTM was preferentially mediated by visual neurons identified by memory-guided saccade. These results indicate that LIP plays a crucial role in the early stage of many-to-one sensorimotor transformation.

Spatial Neuronal Integration Supports a Global Representation of Visual Numerosity in Primate Association Cortices

Our sense of number rests on the activity of neurons that are tuned to the number of items and show great invariance across display formats and modalities. Whether numerosity coding becomes abstracted from local spatial representations characteristic of visual input is not known. We mapped the visual receptive fields (RFs) of numerosity-selective neurons in the pFC and ventral intraparietal area in rhesus monkeys. We found numerosity selectivity in pFC and ventral intraparietal neurons irrespective of whether they exhibited an RF and independent of the location of their RFs. RFs were not predictive of the preference of numerosity-selective neurons. Furthermore, the presence and location of RFs had no impact on tuning width and quality of the numerosity-selective neurons. These findings show that neurons in frontal and parietal cortices integrate abstract visuospatial stimuli to give rise to global and spatially released number representations as required for number perception.

Mechanisms of competitive selection: A canonical neural circuit framework

Competitive selection, the transformation of multiple competing sensory inputs and internal states into a unitary choice, is a fundamental component of animal behavior. Selection behaviors have been studied under several intersecting umbrellas including decision-making, action selection, perceptual categorization, and attentional selection. Neural correlates of these behaviors and computational models have been investigated extensively. However, specific, identifiable neural circuit mechanisms underlying the implementation of selection remain elusive. Here, we employ a first principles approach to map competitive selection explicitly onto neural circuit elements. We decompose selection into six computational primitives, identify demands that their execution places on neural circuit design, and propose a canonical neural circuit framework. The resulting framework has several links to neural literature, indicating its biological feasibility, and has several common elements with prominent computational models, suggesting its generality. We propose that this framework can help catalyze experimental discovery of the neural circuit underpinnings of competitive selection.

Dynamics and Hierarchical Encoding of Non-compact Acoustic Categories in Auditory and Frontal Cortex

Categorical perception is a fundamental cognitive function enabling animals to flexibly assign sounds into behaviorally relevant categories. This study investigates the nature of acoustic category representations, their emergence in an ascending series of ferret auditory and frontal cortical fields, and the dynamics of this representation during passive listening to task-relevant stimuli and during active retrieval from memory while engaging in learned categorization tasks. Ferrets were trained on two auditory Go-NoGo categorization tasks to discriminate two non-compact sound categories (composed of tones or amplitude-modulated noise). Neuronal responses became progressively more categorical in higher cortical fields, especially during task performance. The dynamics of the categorical responses exhibited a cascading top-down modulation pattern that began earliest in the frontal cortex and subsequently flowed downstream to the secondary auditory cortex, followed by the primary auditory cortex. In a subpopulation of neurons, categorical responses persisted even during the passive listening condition, demonstrating memory for task categories and their enhanced categorical boundaries.

Posterior parietal cortex plays a causal role in perceptual and categorical decisions

Posterior parietal cortex (PPC) activity correlates with monkeys' decisions during visual discrimination and categorization tasks. However, recent work has questioned whether decision-correlated PPC activity plays a causal role in such decisions. That study focused on PPC's contribution to motor aspects of decisions (deciding where to move), but not sensory evaluation aspects (deciding what you are looking at). We employed reversible inactivation to compare PPC's contributions to motor and sensory aspects of decisions. Inactivation affected both aspects of behavior, but preferentially impaired decisions when visual stimuli, rather than motor response targets, were in the inactivated visual field. This demonstrates a causal role for PPC in decision-making, with preferential involvement in evaluating attended task-relevant sensory stimuli compared with motor planning.

Category-based attentional capture can be influenced by color- and shape-dimensions independently in the conjunction search task

Attention can be involuntarily attracted by a distractor that matches the current attentional control settings (ACSs). However, it remains unclear whether two category-specific ACSs can operate independently. By defining a target as a combination of two prototype-based categories, the present event-related potential (ERP) study investigated how color-category and shape-category ACSs operate within a search task paradigm and the effects of temporal task demands on these ACSs. The matching level between target and distractor was manipulated to separate the effects of each ACS. The relative position between target and distractor was employed to isolate the attentional processing of the distractor from the target. Furthermore, two display durations were used to manipulate the temporal task demands, including a short fixed window (800 ms) and a dynamic window extended until the user responded. Our results support a two-stage selection scenario. In early stage, the color- and shape-ACS independently guided attention to task-relevant property (N2pc components) and suppressed attention toward task-irrelevant properties (P_D components). In late stage, these two independent ACSs were integrated into a holistic ACS to interfere with the consolidation (contralateral delay activity components) and behavioral performance (accuracy and RTs) of target identification. Moreover, an early N1/P1 component might reflect a preattentive enhancement of relevant information or a preattentive suppression of irrelevant objects. These two category-specific ACSs weights differently in varied temporal task demands. These findings support the idea that independent early processing is followed by integrated late processing, which can be applied to category-based attentional capture with different temporal task demands.

Ensemble representations reveal distinct neural coding of visual working memory

We characterized the population-level neural coding of ensemble representations in visual working memory from human electroencephalography. Ensemble representations provide a unique opportunity to investigate structured representations of working memory because the visual system encodes high-order summary statistics as well as noisy sensory inputs in a hierarchical manner. Here, we consistently observe stable coding of simple features as well as the ensemble mean in frontocentral electrodes, which even correlated with behavioral indices of the ensemble across individuals. In occipitoparietal electrodes, however, we find that remembered features are dynamically coded over time, whereas neural coding of the ensemble mean is absent in the old/new judgment task. In contrast, both dynamic and stable coding are found in the continuous estimation task. Our findings suggest that the prefrontal cortex holds behaviorally relevant abstract representations while visual representations in posterior and visual areas are modulated by the task demands.

Sensory processing and categorization in cortical and deep neural networks

Many recent advances in artificial intelligence (AI) are rooted in visual neuroscience. However, ideas from more complicated paradigms like decision-making are less used. Although automated decision-making systems are ubiquitous (driverless cars, pilot support systems, medical diagnosis algorithms etc.), achieving human-level performance in decision making tasks is still a challenge. At the same time, these tasks that are hard for AI are easy for humans. Thus, understanding human brain dynamics during these decision-making tasks and modeling them using deep neural networks could improve AI performance. Here we modelled some of the complex neural interactions during a sensorimotor decision making task. We investigated how brain dynamics flexibly represented and distinguished between sensory processing and categorization in two sensory domains: motion direction and color. We used two different approaches for understanding neural representations. We compared brain responses to 1) the geometry of a sensory or category domain (domain selectivity) and 2) predictions from deep neural networks (computation selectivity). Both approaches gave us similar results. This confirmed the validity of our analyses. Using the first approach, we found that neural representations changed depending on context. We then trained deep recurrent neural networks to perform the same tasks as the animals. Using the second approach, we found that computations in different brain areas also changed flexibly depending on context. Color computations appeared to rely more on sensory processing, while motion computations more on abstract categories. Overall, our results shed light to the biological basis of categorization and differences in selectivity and computations in different brain areas. They also suggest a way for studying sensory and categorical representations in the brain: compare brain responses to both a behavioral model and a deep neural network and test if they give similar results.

Hierarchical recruitment of competition alleviates working memory overload in a frontoparietal model

The storage limitations of visual working memory have been the subject of intense research interest for several decades, but few studies have systematically investigated the dependence of these limitations on memory load that exceeds our retention abilities. Under this real-world scenario, performance typically declines beyond a critical load among low-performing subjects, a phenomenon known as working memory overload. We used a frontoparietal cortical model to test the hypothesis that high-performing subjects select a manageable number of items for storage, thereby avoiding overload. The model accounts for behavioral and electrophysiological data from high-performing subjects in a parameter regime where competitive encoding in its prefrontal network selects items for storage, interareal projections sustain their representations after stimulus offset, and weak dynamics in its parietal network limit their mutual interference. Violation of these principles accounts for these data among low-performing subjects, implying that poor visual working memory performance reflects poor control over frontoparietal circuitry, making testable predictions for experiments.

Circuit mechanisms for the maintenance and manipulation of information in working memory

Recently it has been proposed that information in working memory (WM) may not always be stored in persistent neuronal activity but can be maintained in 'activity-silent' hidden states, such as synaptic efficacies endowed with short-term synaptic plasticity. To test this idea computationally, we investigated recurrent neural network models trained to perform several WM-dependent tasks, in which WM representation emerges from learning and is not a priori assumed to depend on self-sustained persistent activity. We found that short-term synaptic plasticity can support the short-term maintenance of information, provided that the memory delay period is sufficiently short. However, in tasks that require actively manipulating information, persistent activity naturally emerges from learning, and the amount of persistent activity scales with the degree of manipulation required. These results shed insight into the current debate on WM encoding and suggest that persistent activity can vary markedly between short-term memory tasks with different cognitive demands.

Continuous and discrete representations of feature-based attentional priority in human frontoparietal network

Previous studies suggest that human frontoparietal network represents feature-based attentional priority, yet the precise nature of the priority signals remains unclear. Here, we examined whether priority signals vary continuously or discretely as a function of feature similarity. In an fMRI experiment, we presented two superimposed dot fields moving along two linear directions (leftward and rightward) while varying the angular separation between the two directions. Subjects were cued to attend to one of the two dot fields and respond to a possible speed-up in the cued direction. We used multivariate analysis to evaluate how priority representation of the attended direction changes with feature similarity. We found that in early visual areas as well as posterior intraparietal sulcus and inferior frontal junction, the patterns of neural activity became more different as the feature similarity decreased, indicating a continuous representation of the attended feature. In contrast, patterns of neural activity in anterior intraparietal sulcus and frontal eye field remained invariant to changes in feature similarity, indicating a discrete representation of the attended feature. Such distinct neural coding of attentional priority across the frontoparietal network may make complementary contributions to enable flexible attentional control.