Activity of neurons in cortical area MT during a memory for motion task

Neuronal representation of visual working memory content in the primate primary visual cortex

The human ability to perceive vivid memories as if they "float" before our eyes, even in the absence of actual visual stimuli, captivates the imagination. To determine the neural substrates underlying visual memories, we investigated the neuronal representation of working memory content in the primary visual cortex of monkeys. Our study revealed that neurons exhibit unique responses to different memory contents, using firing patterns distinct from those observed during the perception of external visual stimuli. Moreover, this neuronal representation evolves with alterations in the recalled content and extends beyond the retinotopic areas typically reserved for processing external visual input. These discoveries shed light on the visual encoding of memories and indicate avenues for understanding the remarkable power of the mind's eye.

From innate to instructed: A new look at perceptual decision-making

Understanding how subjects perceive sensory stimuli in their environment and use this information to guide appropriate actions is a major challenge in neuroscience. To study perceptual decision-making in animals, researchers use tasks that either probe spontaneous responses to stimuli (often described as "naturalistic") or train animals to associate stimuli with experimenter-defined responses. Spontaneous decisions rely on animals' pre-existing knowledge, while trained tasks offer greater versatility, albeit often at the cost of extensive training. Here, we review emerging approaches to investigate perceptual decision-making using both spontaneous and trained behaviors, highlighting their strengths and limitations. Additionally, we propose how trained decision-making tasks could be improved to achieve faster learning and a more generalizable understanding of task rules.

A gradual transition from veridical to categorical representations along the visual hierarchy during working memory, but not perception

The ability to stably maintain visual information over brief delays is central to cognitive functioning. One possible way to achieve robust working memory maintenance is by having multiple concurrent mnemonic representations across multiple cortical loci. For example, early visual cortex might contribute to storage by representing information in a "sensory-like" format, while intraparietal sulcus uses a format transformed away from sensory driven responses. As an explicit test of mnemonic code transformations along the visual hierarchy, we quantitatively modeled the progression of veridical-to-categorical orientation representations in human participants. Participants directly viewed, or held in mind, an oriented grating pattern, and the similarity between fMRI activation patterns for different orientations was calculated throughout retinotopic cortex. During direct perception, similarity was clustered around cardinal orientations, while during working memory the obliques were represented more similarly. We modeled these similarity patterns based on the known distribution of orientation information in the natural world: The "veridical" model uses an efficient coding framework to capture hypothesized representations during visual perception. The "categorical" model assumes that different "psychological distances" between orientations result in orientation categorization relative to cardinal axes. During direct perception, the veridical model explained the data well in early visual areas, while the categorical model did worse. During working memory, the veridical model only explained some of the data, while the categorical model gradually gained explanatory power for increasingly anterior retinotopic regions. These findings suggest that directly viewed images are represented veridically, but once visual information is no longer tethered to the sensory world, there is a gradual progression to more categorical mnemonic formats along the visual hierarchy.

Sensory and Choice Responses in MT Distinct from Motion Encoding

The macaque middle temporal (MT) area is well known for its visual motion selectivity and relevance to motion perception, but the possibility of it also reflecting higher-level cognitive functions has largely been ignored. We tested for effects of task performance distinct from sensory encoding by manipulating subjects' temporal evidence-weighting strategy during a direction discrimination task while performing electrophysiological recordings from groups of MT neurons in rhesus macaques (one male, one female). This revealed multiple components of MT responses that were, surprisingly, not interpretable as behaviorally relevant modulations of motion encoding, or as bottom-up consequences of the readout of motion direction from MT. The time-varying motion-driven responses of MT were strongly affected by our strategic manipulation-but with time courses opposite the subjects' temporal weighting strategies. Furthermore, large choice-correlated signals were represented in population activity distinct from its motion responses, with multiple phases that lagged psychophysical readout and even continued after the stimulus (but which preceded motor responses). In summary, a novel experimental manipulation of strategy allowed us to control the time course of readout to challenge the correlation between sensory responses and choices, and population-level analyses of simultaneously recorded ensembles allowed us to identify strong signals that were so distinct from direction encoding that conventional, single-neuron-centric analyses could not have revealed or properly characterized them. Together, these approaches revealed multiple cognitive contributions to MT responses that are task related but not functionally relevant to encoding or decoding of motion for psychophysical direction discrimination, providing a new perspective on the assumed status of MT as a simple sensory area.SIGNIFICANCE STATEMENT This study extends understanding of the middle temporal (MT) area beyond its representation of visual motion. Combining multineuron recordings, population-level analyses, and controlled manipulation of task strategy, we exposed signals that depended on changes in temporal weighting strategy, but did not manifest as feedforward effects on behavior. This was demonstrated by (1) an inverse relationship between temporal dynamics of behavioral readout and sensory encoding, (2) a choice-correlated signal that always lagged the stimulus time points most correlated with decisions, and (3) a distinct choice-correlated signal after the stimulus. These findings invite re-evaluation of MT for functions outside of its established sensory role and highlight the power of experimenter-controlled changes in temporal strategy, coupled with recording and analysis approaches that transcend the single-neuron perspective.

Applying machine learning techniques to detect the deployment of spatial working memory from the spiking activity of MT neurons

Neural signatures of working memory have been frequently identified in the spiking activity of different brain areas. However, some studies reported no memory-related change in the spiking activity of the middle temporal (MT) area in the visual cortex. However, recently it was shown that the content of working memory is reflected as an increase in the dimensionality of the average spiking activity of the MT neurons. This study aimed to find the features that can reveal memory-related changes with the help of machine-learning algorithms. In this regard, different linear and nonlinear features were obtained from the neuronal spiking activity during the presence and absence of working memory. To select the optimum features, the Genetic algorithm, Particle Swarm Optimization, and Ant Colony Optimization methods were employed. The classification was performed using the Support Vector Machine (SVM) and the K-Nearest Neighbor (KNN) classifiers. Our results suggest that the deployment of spatial working memory can be perfectly detected from spiking patterns of MT neurons with an accuracy of 99.65±0.12 using the KNN and 99.50±0.26 using the SVM classifiers.

Perceptual bias contextualized in visually ambiguous stimuli

The visual appearance of an object is a function of stimulus properties as well as perceptual biases imposed by the observer. The context-specific trade-off between both can be measured accurately in a perceptual judgment task, involving grouping by proximity in ambiguous dot lattices. Such grouping depends lawfully on a stimulus parameter of the dot lattices known as their aspect ratio (AR), whose effect is modulated by a perceptual bias representing the preference for a cardinal orientation. In two experiments, we investigated how preceding context can lead to bias modulation, either in a top-down fashion via visual working memory (VWM) or bottom-up via sensory priming. In Experiment 1, we embedded the perceptual judgment task in a change detection paradigm and studied how the factors of VWM load (complexity of the memory array) and content (congruency in orientation to the ensuing dot lattice) affect the prominence of perceptual bias. A robust vertical orientation bias was observed, which was increased by VWM load and modulated by congruent VWM content. In Experiment 2, dot lattices were preceded by oriented primes. Here, primes regardless of orientation elicited a vertical orientation bias in dot lattices compared to a neutral baseline. Taken together, the two experiments demonstrate that top-down context (VWM load and content) effectively controls orientation bias modulation, while bottom-up context (i.e., priming) merely acts as an undifferentiated trigger to perceptual bias. These findings characterize the temporal context sensitivity of Gestalt perception, shed light on the processes responsible for different perceptual outcomes of ambiguous stimuli, and identify some of the mechanisms controlling perceptual bias.

Dynamic and stable population coding of attentional instructions coexist in the prefrontal cortex

A large body of recent work suggests that neural representations in prefrontal cortex (PFC) are changing over time to adapt to task demands. However, it remains unclear whether and how such dynamic coding schemes depend on the encoded variable and are influenced by anatomical constraints. Using a cued attention task and multivariate classification methods, we show that neuronal ensembles in PFC encode and retain in working memory spatial and color attentional instructions in an anatomically specific manner. Spatial instructions could be decoded both from the frontal eye field (FEF) and the ventrolateral PFC (vlPFC) population, albeit more robustly from FEF, whereas color instructions were decoded more robustly from vlPFC. Decoding spatial and color information from vlPFC activity in the high-dimensional state space indicated stronger dynamics for color, across the cue presentation and memory periods. The change in the color code was largely due to rapid changes in the network state during the transition to the delay period. However, we found that dynamic vlPFC activity contained time-invariant color information within a low-dimensional subspace of neural activity that allowed for stable decoding of color across time. Furthermore, spatial attention influenced decoding of stimuli features profoundly in vlPFC, but less so in visual area V4. Overall, our results suggest that dynamic population coding of attentional instructions within PFC is shaped by anatomical constraints and can coexist with stable subspace coding that allows time-invariant decoding of information about the future target.

Reward and loss incentives improve spatial working memory by shaping trial-by-trial posterior frontoparietal signals

Integrating motivational signals with cognition is critical for goal-directed activities. The mechanisms that link neural changes with motivated working memory continue to be understood. Here, we tested how externally cued and non-cued (internally represented) reward and loss impact spatial working memory precision and neural circuits in human subjects using fMRI. We translated the classic delayed-response spatial working memory paradigm from non-human primate studies to take advantage of a continuous numeric measure of working memory precision, and the wealth of translational neuroscience yielded by these studies. Our results demonstrated that both cued and non-cued reward and loss improved spatial working memory precision. Visual association regions of the posterior prefrontal and parietal cortices, specifically the precentral sulcus (PCS) and intraparietal sulcus (IPS), had increased BOLD signal during incentivized spatial working memory. A subset of these regions had trial-by-trial increases in BOLD signal that were associated with better working memory precision, suggesting that these regions may be critical for linking neural signals with motivated working memory. In contrast, regions straddling executive networks, including areas in the dorsolateral prefrontal cortex, anterior parietal cortex and cerebellum displayed decreased BOLD signal during incentivized working memory. While reward and loss similarly impacted working memory processes, they dissociated during feedback when money won or avoided in loss was given based on working memory performance. During feedback, the trial-by-trial amount and valence of reward/loss received was dissociated amongst regions such as the ventral striatum, habenula and periaqueductal gray. Overall, this work suggests motivated spatial working memory is supported by complex sensory processes, and that the IPS and PCS in the posterior frontoparietal cortices may be key regions for integrating motivational signals with spatial working memory precision.

Common and independent processing of visual motion perception and oculomotor response

Visual motion signals are used not only to drive motion perception but also to elicit oculomotor responses. A fundamental question is whether perceptual and oculomotor processing of motion signals shares a common mechanism. This study aimed to address this question using visual motion priming, in which the perceived direction of a directionally ambiguous stimulus is biased in the same (positive priming) or opposite (negative priming) direction as that of a priming stimulus. The priming effect depends on the duration of the priming stimulus. It is assumed that positive and negative priming are mediated by high- and low-level motion systems, respectively. Participants were asked to judge the perceived direction of a π-phase-shifted test grating after a smoothly drifting priming grating during varied durations. Their eye movements were measured while the test grating was presented. The perception and eye movements were discrepant under positive priming and correlated under negative priming on a trial-by-trial basis when an interstimulus interval was inserted between the priming and test stimuli, indicating that the eye movements were evoked by the test stimulus per se. These findings suggest that perceptual and oculomotor responses are induced by a common mechanism at a low level of motion processing but by independent mechanisms at a high level of motion processing.

Neural Substrates of Visual Perception and Working Memory: Two Sides of the Same Coin or Two Different Coins?

Visual perception occurs when a set of physical signals emanating from the environment enter the visual system and the brain interprets such signals as a percept. Visual working memory occurs when the brain produces and maintains a mental representation of a percept while the physical signals corresponding to that percept are not available. Early studies in humans and non-human primates demonstrated that lesions of the prefrontal cortex impair performance during visual working memory tasks but not during perceptual tasks. These studies attributed a fundamental role in working memory and a lesser role in visual perception to the prefrontal cortex. Indeed, single cell recording studies have found that neurons in the lateral prefrontal cortex of macaques encode working memory representations via persistent firing, validating the results of lesion studies. However, other studies have reported that neurons in some areas of the parietal and temporal lobe-classically associated with visual perception-similarly encode working memory representations via persistent firing. This prompted a line of enquiry about the role of the prefrontal and other associative cortices in working memory and perception. Here, we review evidence from single neuron studies in macaque monkeys examining working memory representations across different areas of the visual hierarchy and link them to studies examining the role of the same areas in visual perception. We conclude that neurons in early visual areas of both ventral (V1-V2-V4) and dorsal (V1-V3-MT) visual pathways of macaques mainly encode perceptual signals. On the other hand, areas downstream from V4 and MT contain subpopulations of neurons that encode both perceptual and/or working memory signals. Differences in cortical architecture (neuronal types, layer composition, and synaptic density and distribution) may be linked to the differential encoding of perceptual and working memory signals between early visual areas and higher association areas.

Linking Neuronal Direction Selectivity to Perceptual Decisions About Visual Motion

Psychophysical and neurophysiological studies of responses to visual motion have converged on a consistent set of general principles that characterize visual processing of motion information. Both types of approaches have shown that the direction and speed of target motion are among the most important encoded stimulus properties, revealing many parallels between psychophysical and physiological responses to motion. Motivated by these parallels, this review focuses largely on more direct links between the key feature of the neuronal response to motion, direction selectivity, and its utilization in memory-guided perceptual decisions. These links were established during neuronal recordings in monkeys performing direction discriminations, but also by examining perceptual effects of widespread elimination of cortical direction selectivity produced by motion deprivation during development. Other approaches, such as microstimulation and lesions, have documented the importance of direction-selective activity in the areas that are active during memory-guided direction comparisons, area MT and the prefrontal cortex, revealing their likely interactions during behavioral tasks.

Persistent Activity During Working Memory From Front to Back

Working memory (WM) extends the duration over which information is available for processing. Given its importance in supporting a wide-array of high level cognitive abilities, uncovering the neural mechanisms that underlie WM has been a primary goal of neuroscience research over the past century. Here, we critically review what we consider the two major "arcs" of inquiry, with a specific focus on findings that were theoretically transformative. For the first arc, we briefly review classic studies that led to the canonical WM theory that cast the prefrontal cortex (PFC) as a central player utilizing persistent activity of neurons as a mechanism for memory storage. We then consider recent challenges to the theory regarding the role of persistent neural activity. The second arc, which evolved over the last decade, stemmed from sophisticated computational neuroimaging approaches enabling researchers to decode the contents of WM from the patterns of neural activity in many parts of the brain including early visual cortex. We summarize key findings from these studies, their implications for WM theory, and finally the challenges these findings pose. Our goal in doing so is to identify barriers to developing a comprehensive theory of WM that will require a unification of these two "arcs" of research.

Delayed-matching-to-position working memory in mice relies on NMDA-receptors in prefrontal pyramidal cells

A hypofunction of N-methyl-D-aspartate glutamate receptors (NMDARs) has been implicated in the pathogenesis of schizophrenia by clinical and rodent studies. However, to what extent NMDAR-hypofunction in distinct cell-types across the brain causes different symptoms of this disease is largely unknown. One pharmaco-resistant core symptom of schizophrenia is impaired working memory (WM). NMDARs have been suggested to mediate sustained firing in excitatory neurons of the prefrontal cortex (PFC) that might underlie WM storage. However, if NMDAR-hypofunction in prefrontal excitatory neurons may indeed entail WM impairments is unknown. We here investigated this question in mice, in which NMDARs were genetically-ablated in PFC excitatory cells. This cell type-selective NMDAR-hypofunction caused a specific deficit in a delayed-matching-to-position (DMTP) 5-choice-based operant WM task. In contrast, T-maze rewarded alternation and several psychological functions including attention, spatial short-term habituation, novelty-processing, motivation, sociability, impulsivity, and hedonic valuation remained unimpaired at the level of GluN1-hypofunction caused by our manipulation. Our data suggest that a hypofunction of NMDARs in prefrontal excitatory neurons may indeed cause WM impairments, but are possibly not accounting for most other deficits in schizophrenia.

Explaining short-term memory phenomena with an integrated episodic/semantic framework of long-term memory

Current thinking about human memory is dominated by distinctions between episodic and semantic memory and between short-term memory (STM) and long-term memory (LTM). However, many memory phenomena seem to cut across these distinctions. This article attempts to set the groundwork for the issues that need to be resolved in generating an integrated model of long-term memory that incorporates semantic, episodic, and short-term memory. We contrast Nairne's (2002, Annual Review of Psychology) consensus account of short-term memory with a relatively generic theory of an integrated episodic-semantic memory. The later consists primarily of a list of principles which we and others argue are necessary to include in any theory of long-term memory. We then add some more specific assumptions to outline a modern theory of forgetting. We then turn to the issue of much of the phenomena thought to necessitate a dedicated short-term memory can be explained by an integrated theory of episodic and semantic memory. Our conclusion is that an integrated theory of long-term memory must be augmented to explain a small number of outstanding memory phenomena. Finally, we ask whether the augmentation needs to involve a dedicated mnemonic system, or sensory or language-based systems, which also have mnemonic capabilities.

Expansion and Contraction Modulate Visual Short-Term Memory

We examined the influence of expansion and contraction on visual short-term memory (VSTM) using a change-detection task. In each trial, several expanding/contracting line segments with various orientations were presented in two successive displays. The orientation of objects in the second display was either identical to, or different from, that of the first display. Observers were asked to judge the presence or absence of an orientation change in successive displays. Results showed that memory performance for expanding objects was higher than for contracting objects (expansion benefit: Experiments 1A and 1B). Further experiments focused on VSTM processing (encoding, storage, and retrieval). Regarding the retrieval stage, an expansion benefit was replicated only when the direction of motion was consistent between two successive displays (Experiment 2A). A cueing stimulus enhanced the memory performance for both expanding and contracting motions and eliminated the expansion benefit (Experiment 2B). Regarding the storage stage, we found the expansion benefit occurred only for shorter blank intervals between the two displays (Experiment 3). Regarding the encoding stage, the expansion benefit was observed regardless of presentation times (Experiment 4). These results indicate the possibility that expanding and contracting motions modulate VSTM.

Autocorrelation Structure in the Macaque Dorsolateral, But not Orbital or Polar, Prefrontal Cortex Predicts Response-Coding Strength in a Visually Cued Strategy Task

In previous work, we studied the activity of neurons in the dorsolateral (PFdl), orbital (PFo), and polar (PFp) prefrontal cortex while monkeys performed a strategy task with 2 spatial goals. A cue instructed 1 of 2 strategies in each trial: stay with the previous goal or shift to the alternative goal. Each trial started with a fixation period, followed by a cue. Subsequently, a delay period was followed by a "go" signal that instructed the monkeys to choose one goal. After each choice, feedback was provided. In this study, we focused on the temporal receptive fields of the neurons, as measured by the decay in autocorrelation (time constant) during the fixation period, and examined the relationship with response and strategy coding. The temporal receptive field in PFdl correlated with the response-related but not with the strategy-related modulation in the delay and the feedback periods: neurons with longer time constants in PFdl tended to show stronger and more prolonged response coding. No such correlation was found in PFp or PFo. These findings demonstrate that the temporal specialization of neurons for temporally extended computations is predictive of response coding, and neurons in PFdl, but not PFp or PFo, develop such predictive properties.

Reevaluating the Role of Persistent Neural Activity in Short-Term Memory

A traditional view of short-term working memory (STM) is that task-relevant information is maintained 'online' in persistent spiking activity. However, recent experimental and modeling studies have begun to question this long-held belief. In this review, we discuss new evidence demonstrating that information can be 'silently' maintained via short-term synaptic plasticity (STSP) without the need for persistent activity. We discuss how the neural mechanisms underlying STM are inextricably linked with the cognitive demands of the task, such that the passive maintenance and the active manipulation of information are subserved differently in the brain. Together, these recent findings point towards a more nuanced view of STM in which multiple substrates work in concert to support our ability to temporarily maintain and manipulate information.

Activity in LIP, But not V4, Matches Performance When Attention is Spread

The enhancement of neuronal responses in many visual areas while animals perform spatial attention tasks has widely been thought to be the neural correlate of visual attention, but it is unclear whether the presence or absence of this modulation contributes to our striking inability to notice changes in change blindness examples. We asked whether neuronal responses in visual area V4 and the lateral intraparietal area (LIP) in posterior parietal cortex could explain the limited ability of subjects to attend multiple items in a display. We trained animals to perform a change detection task in which they had to compare 2 arrays of stimuli separated briefly in time and found that each animal's performance decreased as function of set-size. Neuronal discriminability in V4 was consistent across set-sizes, but decreased for higher set-sizes in LIP. The introduction of a reward bias produced attentional enhancement in V4, but this could not explain the vast improvement in performance, whereas the enhancement in LIP responses could. We suggest that behavioral set-size effects and the marked improvement in performance with focused attention may not be related to response enhancement in V4 but, instead, may occur in or on the way to LIP.

Perturbing Neural Representations of Working Memory with Task-irrelevant Interruption

Working memory maintains information so that it can be used in complex cognitive tasks. A key challenge for this system is to maintain relevant information in the face of task-irrelevant perturbations. Across two experiments, we investigated the impact of task-irrelevant interruptions on neural representations of working memory. We recorded EEG activity in humans while they performed a working memory task. On a subset of trials, we interrupted participants with salient but task-irrelevant objects. To track the impact of these task-irrelevant interruptions on neural representations of working memory, we measured two well-characterized, temporally sensitive EEG markers that reflect active, prioritized working memory representations: the contralateral delay activity and lateralized alpha power (8-12 Hz). After interruption, we found that contralateral delay activity amplitude momentarily sustained but was gone by the end of the trial. Lateralized alpha power was immediately influenced by the interrupters but recovered by the end of the trial. This suggests that dissociable neural processes contribute to the maintenance of working memory information and that brief irrelevant onsets disrupt two distinct online aspects of working memory. In addition, we found that task expectancy modulated the timing and magnitude of how these two neural signals responded to task-irrelevant interruptions, suggesting that the brain's response to task-irrelevant interruption is shaped by task context.

Coexisting representations of sensory and mnemonic information in human visual cortex

Traversing sensory environments requires keeping relevant information in mind while simultaneously processing new inputs. Visual information is kept in working memory via feature selective responses in early visual cortex, but recent work had suggested that new sensory inputs obligatorily wipe out this information. Here we show region-wide multiplexing abilities in classic sensory areas, with population-level response patterns in early visual cortex representing the contents of working memory alongside new sensory inputs. In a second experiment, we show that when people get distracted, this leads to both disruptions of mnemonic information in early visual cortex and decrements in behavioral recall. Representations in the intraparietal sulcus reflect actively remembered information encoded in a transformed format, but not task-irrelevant sensory inputs. Together these results suggest that early visual areas play a key role in supporting high resolution working memory representations that can serve as a template for comparing incoming sensory information.

Behavioral and oculomotor evidence for visual simulation of object movement

We regularly interact with moving objects in our environment. Yet, little is known about how we extrapolate the future movements of visually perceived objects. One possibility is that movements are experienced by a mental visual simulation, allowing one to internally picture an object's upcoming motion trajectory, even as the object itself remains stationary. Here we examined this possibility by asking human participants to make judgments about the future position of a falling ball on an obstacle-filled display. We found that properties of the ball's trajectory were highly predictive of subjects' reaction times and accuracy on the task. We also found that the eye movements subjects made while attempting to ascertain where the ball might fall had significant spatiotemporal overlap with those made while actually perceiving the ball fall. These findings suggest that subjects simulated the ball's trajectory to inform their responses. Finally, we trained a convolutional neural network to see whether this problem could be solved by simple image analysis as opposed to the more intricate simulation strategy we propose. We found that while the network was able to solve our task, the model's output did not effectively or consistently predict human behavior. This implies that subjects employed a different strategy for solving our task, and bolsters the conclusion that they were engaging in visual simulation. The current study thus provides support for visual simulation of motion as a means of understanding complex visual scenes and paves the way for future investigations of this phenomenon at a neural level.

Spiking Suppression Precedes Cued Attentional Enhancement of Neural Responses in Primary Visual Cortex

Attending to a visual stimulus increases its detectability, even if gaze is directed elsewhere. This covert attentional selection is known to enhance spiking across many brain areas, including the primary visual cortex (V1). Here we investigate the temporal dynamics of attention-related spiking changes in V1 of macaques performing a task that separates attentional selection from the onset of visual stimulation. We found that preceding attentional enhancement there was a sharp, transient decline in spiking following presentation of an attention-guiding cue. This disruption of V1 spiking was not observed in a task-naïve subject that passively observed the same stimulus sequence, suggesting that sensory activation is insufficient to cause suppression. Following this suppression, attended stimuli evoked more spiking than unattended stimuli, matching previous reports of attention-related activity in V1. Laminar analyses revealed a distinct pattern of activation in feedback-associated layers during both the cue-induced suppression and subsequent attentional enhancement. These findings suggest that top-down modulation of V1 spiking can be bidirectional and result in either suppression or enhancement of spiking responses.

Neural Intrinsic Timescales in the Macaque Dorsal Premotor Cortex Predict the Strength of Spatial Response Coding

Our brain continuously receives information over multiple timescales that are differently processed across areas. In this study, we investigated the intrinsic timescale of neurons in the dorsal premotor cortex (PMd) of two rhesus macaques while performing a non-match-to-goal task. The task rule was to reject the previously chosen target and select the alternative one. We defined the intrinsic timescale as the decay constant of the autocorrelation structure computed during a baseline period of the task. We found that neurons with longer intrinsic timescale tended to maintain a stronger spatial response coding during a delay period. This result suggests that longer intrinsic timescales predict the functional role of PMd neurons in a cognitive task. Our estimate of the intrinsic timescale integrates an existing hierarchical model (Murray et al., 2014), by assigning to PMd a lower position than prefrontal cortex in the hierarchical ordering of the brain areas based on neurons' timescales.

•

The spatial response encoding during a delay depends on neurons' timescales

•

Longer intrinsic timescales foretell the role of PMd neurons in a cognitive task

•

PMd occupies a lower level than PF in the hierarchical organization of brain areas

Selective attention on representations in working memory: cognitive and neural mechanisms

Selective attention and working memory are inter-dependent core cognitive functions. It is critical to allocate attention on selected targets during the capacity-limited working memory processes to fulfill the goal-directed behavior. The trends of research on both topics are increasing exponentially in recent years, and it is considered that selective attention and working memory share similar underlying neural mechanisms. Different types of attention orientation in working memory are introduced by distinctive cues, and the means using retrospective cues are strengthened currently as it is manipulating the representation in memory, instead of the perceptual representation. The cognitive and neural mechanisms of the retro-cue effects are further reviewed, as well as the potential molecular mechanism. The frontal-parietal network that is involved in both attention and working memory is also the neural candidate for attention orientation during working memory. Neural oscillations in the gamma and alpha/beta oscillations may respectively be employed for the feedforward and feedback information transfer between the sensory cortices and the association cortices. Dopamine and serotonin systems might interact with each other subserving the communication between memory and attention. In conclusion, representations which attention shifts towards are strengthened, while representations which attention moves away from are degraded. Studies on attention orientation during working memory indicates the flexibility of the processes of working memory, and the beneficial way that overcome the limited capacity of working memory.

Reevaluating the Sensory Account of Visual Working Memory Storage

Recent human fMRI pattern-decoding studies have highlighted the involvement of sensory areas in visual working memory (VWM) tasks and argue for a sensory account of VWM storage. In this review, evidence is examined from human behavior, fMRI decoding, and transcranial magnetic stimulation (TMS) studies, as well as from monkey neurophysiology studies. Contrary to the prevalent view, the available evidence provides little support for the sensory account of VWM storage. Instead, when the ability to resist distraction and the existence of top-down feedback are taken into account, VWM-related activities in sensory areas seem to reflect feedback signals indicative of VWM storage elsewhere in the brain. Collectively, the evidence shows that prefrontal and parietal regions, rather than sensory areas, play more significant roles in VWM storage.

Mechanisms of Persistent Activity in Cortical Circuits: Possible Neural Substrates for Working Memory

A commonly observed neural correlate of working memory is firing that persists after the triggering stimulus disappears. Substantial effort has been devoted to understanding the many potential mechanisms that may underlie memory-associated persistent activity. These rely either on the intrinsic properties of individual neurons or on the connectivity within neural circuits to maintain the persistent activity. Nevertheless, it remains unclear which mechanisms are at play in the many brain areas involved in working memory. Herein, we first summarize the palette of different mechanisms that can generate persistent activity. We then discuss recent work that asks which mechanisms underlie persistent activity in different brain areas. Finally, we discuss future studies that might tackle this question further. Our goal is to bridge between the communities of researchers who study either single-neuron biophysical, or neural circuit, mechanisms that can generate the persistent activity that underlies working memory.

Sustained Activity Encoding Working Memories: Not Fully Distributed

Working memory (WM) is the ability to remember and manipulate information for short time intervals. Recent studies have proposed that sustained firing encoding the contents of WM is ubiquitous across cortical neurons. We review here the collective evidence supporting this claim. A variety of studies report that neurons in prefrontal, parietal, and inferotemporal association cortices show robust sustained activity encoding the location and features of memoranda during WM tasks. However, reports of WM-related sustained activity in early sensory areas are rare, and typically lack stimulus specificity. We propose that robust sustained activity that can support WM coding arises as a property of association cortices downstream from the early stages of sensory processing.

Spatial working memory alters the efficacy of input to visual cortex

Prefrontal cortex modulates sensory signals in extrastriate visual cortex, in part via its direct projections from the frontal eye field (FEF), an area involved in selective attention. We find that working memory-related activity is a dominant signal within FEF input to visual cortex. Although this signal alone does not evoke spiking responses in areas V4 and MT during memory, the gain of visual responses in these areas increases, and neuronal receptive fields expand and shift towards the remembered location, improving the stimulus representation by neuronal populations. These results provide a basis for enhancing the representation of working memory targets and implicate persistent FEF activity as a basis for the interdependence of working memory and selective attention.

Frontal eye field (FEF) is a visual prefrontal area involved in top-down attention. Here the authors report that FEF neurons projecting to V4/MT are persistently active during spatial working memory, and V4/MT neurons show changes in receptive field and gain at the location held in working memory.

The Distributed Nature of Working Memory

Studies in humans and non-human primates have provided evidence for storage of working memory contents in multiple regions ranging from sensory to parietal and prefrontal cortex. We discuss potential explanations for these distributed representations: (i) features in sensory regions versus prefrontal cortex differ in the level of abstractness and generalizability; and (ii) features in prefrontal cortex reflect representations that are transformed for guidance of upcoming behavioral actions. We propose that the propensity to produce persistent activity is a general feature of cortical networks. Future studies may have to shift focus from asking where working memory can be observed in the brain to how a range of specialized brain areas together transform sensory information into a delayed behavioral response.

Neural mechanisms of information storage in visual short-term memory

The capacity to briefly memorize fleeting sensory information supports visual search and behavioral interactions with relevant stimuli in the environment. Traditionally, studies investigating the neural basis of visual short term memory (STM) have focused on the role of prefrontal cortex (PFC) in exerting executive control over what information is stored and how it is adaptively used to guide behavior. However, the neural substrates that support the actual storage of content-specific information in STM are more controversial, with some attributing this function to PFC and others to the specialized areas of early visual cortex that initially encode incoming sensory stimuli. In contrast to these traditional views, I will review evidence suggesting that content-specific information can be flexibly maintained in areas across the cortical hierarchy ranging from early visual cortex to PFC. While the factors that determine exactly where content-specific information is represented are not yet entirely clear, recognizing the importance of task-demands and better understanding the operation of non-spiking neural codes may help to constrain new theories about how memories are maintained at different resolutions, across different timescales, and in the presence of distracting information.

Discrete capacity limits and neuroanatomical correlates of visual short-term memory for objects and spatial locations

Working memory is responsible for keeping information in mind when it is no longer in view, linking perception with higher cognitive functions. Despite such crucial role, short-term maintenance of visual information is severely limited. Research suggests that capacity limits in visual short-term memory (VSTM) are correlated with sustained activity in distinct brain areas. Here, we investigated whether variability in the structure of the brain is reflected in individual differences of behavioral capacity estimates for spatial and object VSTM. Behavioral capacity estimates were calculated separately for spatial and object information using a novel adaptive staircase procedure and were found to be unrelated, supporting domain-specific VSTM capacity limits. Voxel-based morphometry (VBM) analyses revealed dissociable neuroanatomical correlates of spatial versus object VSTM. Interindividual variability in spatial VSTM was reflected in the gray matter density of the inferior parietal lobule. In contrast, object VSTM was reflected in the gray matter density of the left insula. These dissociable findings highlight the importance of considering domain-specific estimates of VSTM capacity and point to the crucial brain regions that limit VSTM capacity for different types of visual information. Hum Brain Mapp 38:767-778, 2017. © 2016 Wiley Periodicals, Inc.

Turning Symbolic: The Representation of Motion Direction in Working Memory

What happens to the representation of a moving stimulus when it is no longer present and its motion direction has to be maintained in working memory (WM)? Is the initial, sensorial representation maintained during the delay period or is there another representation, at a higher level of abstraction? It is also feasible that multiple representations may co-exist in WM, manifesting different facets of sensory and more abstract features. To that end, we investigated the mnemonic representation of motion direction in a series of three psychophysical experiments, using a delayed motion-discrimination task (relative clockwise∖counter-clockwise judgment). First, we show that a change in the dots’ contrast polarity does not hamper performance. Next, we demonstrate that performance is unaffected by relocation of the Test stimulus in either retinotopic or spatiotopic coordinate frames. Finally, we show that an arrow-shaped cue presented during the delay interval between the Sample and Test stimulus, strongly biases performance toward the direction of the arrow, although the cue itself is non-informative (it has no predictive value of the correct answer). These results indicate that the representation of motion direction in WM could be independent of the physical features of the stimulus (polarity or position) and has non-sensorial abstract qualities. It is plausible that an abstract mnemonic trace might be activated alongside a more basic, analog representation of the stimulus. We speculate that the specific sensitivity of the mnemonic representation to the arrow-shaped symbol may stem from the long term learned association between direction and the hour in the clock.

Multi-Voxel Decoding and the Topography of Maintained Information During Visual Working Memory

The ability to maintain representations in the absence of external sensory stimulation, such as in working memory, is critical for guiding human behavior. Human functional brain imaging studies suggest that visual working memory can recruit a network of brain regions from visual to parietal to prefrontal cortex. In this review, we focus on the maintenance of representations during visual working memory and discuss factors determining the topography of those representations. In particular, we review recent studies employing multi-voxel pattern analysis (MVPA) that demonstrate decoding of the maintained content in visual cortex, providing support for a “sensory recruitment” model of visual working memory. However, there is some evidence that maintained content can also be decoded in areas outside of visual cortex, including parietal and frontal cortex. We suggest that the ability to maintain representations during working memory is a general property of cortex, not restricted to specific areas, and argue that it is important to consider the nature of the information that must be maintained. Such information-content is critically determined by the task and the recruitment of specific regions during visual working memory will be both task- and stimulus-dependent. Thus, the common finding of maintained information in visual, but not parietal or prefrontal, cortex may be more of a reflection of the need to maintain specific types of visual information and not of a privileged role of visual cortex in maintenance.

Overwriting and intrusion in short-term memory

Studies of interference in working and short-term memory suggest that irrelevant information may overwrite the contents of memory or intrude into memory. While some previous studies have reported greater interference when irrelevant information is similar to the contents of memory than when it is dissimilar, other studies have reported greater interference for dissimilar distractors than for similar distractors. In the present study, we find the latter effect in a paradigm that uses auditory tones as stimuli. We suggest that the effects of distractor similarity to memory contents are mediated by the type of information held in memory, particularly the complexity or simplicity of information.

Unilateral prefrontal lesions impair memory-guided comparisons of contralateral visual motion

The contribution of the lateral prefrontal cortex (LPFC) to working memory is the topic of active debate. On the one hand, it has been argued that the persistent delay activity in LPFC recorded during some working memory tasks is a reflection of sensory storage, the notion supported by some lesion studies. On the other hand, there is emerging evidence that the LPFC plays a key role in the maintenance of sensory information not by storing relevant visual signals but by allocating visual attention to such stimuli. In this study, we addressed this question by examining the effects of unilateral LPFC lesions during a working memory task requiring monkeys to compare directions of two moving stimuli, separated by a delay. The lesions resulted in impaired thresholds for contralesional stimuli at longer delays, and these deficits were most dramatic when the task required rapid reallocation of spatial attention. In addition, these effects were equally pronounced when the remembered stimuli were at threshold or moved coherently. The contralesional nature of the deficits points to the importance of the interactions between the LPFC and the motion processing neurons residing in extrastriate area MT. Delay-specificity of the deficit supports LPFC involvement in the maintenance stage of the comparison task. However, because this deficit was independent of stimulus features giving rise to the remembered direction and was most pronounced during rapid shifts of attention, its role is more likely to be attending and accessing the preserved motion signals rather than their storage.

The influence of spatial pattern on visual short-term memory for contrast

Several psychophysical studies of visual short-term memory (VSTM) have shown high-fidelity storage capacity for many properties of visual stimuli. On judgments of the spatial frequency of gratings, for example, discrimination performance does not decrease significantly, even for memory intervals of up to 30 s. For other properties, such as stimulus orientation and contrast, however, such “perfect storage” behavior is not found, although the reasons for this difference remain unresolved. Here, we report two experiments in which we investigated the nature of the representation of stimulus contrast in VSTM using spatially complex, two-dimensional random-noise stimuli. We addressed whether information about contrast per se is retained during the memory interval by using a test stimulus with the same spatial structure but either the same or the opposite local contrast polarity, with respect to the comparison (i.e., remembered) stimulus. We found that discrimination thresholds got steadily worse with increasing duration of the memory interval. Furthermore, performance was better when the test and comparison stimuli had the same local contrast polarity than when they were contrast-reversed. Finally, when a noise mask was introduced during the memory interval, its disruptive effect was maximal when the spatial configuration of its constituent elements was uncorrelated with those of the comparison and test stimuli. These results suggest that VSTM for contrast is closely tied to the spatial configuration of stimuli and is not transformed into a more abstract representation.

Prefrontal cortex and sensory cortices during working memory: quantity and quality

The activity in sensory cortices and the prefrontal cortex (PFC) throughout the delay interval of working memory (WM) tasks reflect two aspects of WM-quality and quantity, respectively. The delay activity in sensory cortices is fine-tuned to sensory information and forms the neural basis of the precision of WM storage, while the delay activity in the PFC appears to represent behavioral goals and filters out irrelevant distractions, forming the neural basis of the quantity of task-relevant information in WM. The PFC and sensory cortices interact through different frequency bands of neuronal oscillation (theta, alpha, and gamma) to fulfill goal-directed behaviors.

A hierarchy of intrinsic timescales across primate cortex

Specialization and hierarchy are organizing principles for primate cortex, yet there is little direct evidence for how cortical areas are specialized in the temporal domain. We measured timescales of intrinsic fluctuations in spiking activity across areas and found a hierarchical ordering, with sensory and prefrontal areas exhibiting shorter and longer timescales, respectively. On the basis of our findings, we suggest that intrinsic timescales reflect areal specialization for task-relevant computations over multiple temporal ranges.

Ghosts in the machine: memory interference from the previous trial

Previous memoranda can interfere with the memorization or storage of new information, a concept known as proactive interference. Studies of proactive interference typically use categorical memoranda and match-to-sample tasks with categorical measures such as the proportion of correct to incorrect responses. In this study we instead train five macaques in a spatial memory task with continuous memoranda and responses, allowing us to more finely probe working memory circuits. We first ask whether the memoranda from the previous trial result in proactive interference in an oculomotor delayed response task. We then characterize the spatial and temporal profile of this interference and ask whether this profile can be predicted by an attractor network model of working memory. We find that memory in the current trial shows a bias toward the location of the memorandum of the previous trial. The magnitude of this bias increases with the duration of the memory period within which it is measured. Our simulations using standard attractor network models of working memory show that these models easily replicate the spatial profile of the bias. However, unlike the behavioral findings, these attractor models show an increase in bias with the duration of the previous rather than the current memory period. To model a bias that increases with current trial duration we posit two separate memory stores, a rapidly decaying visual store that resists proactive interference effects and a sustained memory store that is susceptible to proactive interference.

Sharp emergence of feature-selective sustained activity along the dorsal visual pathway

Sustained activity encoding visual working memory representations has been observed in several cortical areas of primates. Where along the visual pathways this activity emerges remains unknown. Here we show in macaques that sustained spiking activity encoding memorized visual motion directions is absent in direction-selective neurons in early visual area middle temporal (MT). However, it is robustly present immediately downstream, in multimodal association area medial superior temporal (MST), and in the lateral prefrontal cortex (LPFC). This sharp emergence of sustained activity along the dorsal pathway suggests a functional boundary between early visual areas, encoding sensory inputs, and downstream association areas, additionally encoding mnemonic representations. Moreover, local field potential oscillations in MT encoded the memorized directions and, in the low frequencies, were phase-coherent with LPFC spikes. This suggests that LPFC sustained activity modulates synaptic activity in MT, a putative top-down mechanism by which memory signals influence stimulus processing in early visual cortex.

Human area MT+ shows load-dependent activation during working memory maintenance with continuously morphing stimulation

Background

Initially, human area MT⁺ was considered a visual area solely processing motion information but further research has shown that it is also involved in various different cognitive operations, such as working memory tasks requiring motion-related information to be maintained or cognitive tasks with implied or expected motion.

In the present fMRI study in humans, we focused on MT⁺ modulation during working memory maintenance using a dynamic shape-tracking working memory task with no motion-related working memory content. Working memory load was systematically varied using complex and simple stimulus material and parametrically increasing retention periods. Activation patterns for the difference between retention of complex and simple memorized stimuli were examined in order to preclude that the reported effects are caused by differences in retrieval.

Results

Conjunction analysis over all delay durations for the maintenance of complex versus simple stimuli demonstrated a wide-spread activation pattern. Percent signal change (PSC) in area MT⁺ revealed a pattern with higher values for the maintenance of complex shapes compared to the retention of a simple circle and with higher values for increasing delay durations.

Conclusions

The present data extend previous knowledge by demonstrating that visual area MT⁺ presents a brain activity pattern usually found in brain regions that are actively involved in working memory maintenance.

Homeostatic regulation of memory systems and adaptive decisions

While it is clear that many brain areas process mnemonic information, understanding how their interactions result in continuously adaptive behaviors has been a challenge. A homeostatic-regulated prediction model of memory is presented that considers the existence of a single memory system that is based on a multilevel coordinated and integrated network (from cells to neural systems) that determines the extent to which events and outcomes occur as predicted. The “multiple memory systems of the brain” have in common output that signals errors in the prediction of events and/or their outcomes, although these signals differ in terms of what the error signal represents (e.g., hippocampus: context prediction errors vs. midbrain/striatum: reward prediction errors). The prefrontal cortex likely plays a pivotal role in the coordination of prediction analysis within and across prediction brain areas. By virtue of its widespread control and influence, and intrinsic working memory mechanisms. Thus, the prefrontal cortex supports the flexible processing needed to generate adaptive behaviors and predict future outcomes. It is proposed that prefrontal cortex continually and automatically produces adaptive responses according to homeostatic regulatory principles: prefrontal cortex may serve as a controller that is intrinsically driven to maintain in prediction areas an experience-dependent firing rate set point that ensures adaptive temporally and spatially resolved neural responses to future prediction errors. This same drive by prefrontal cortex may also restore set point firing rates after deviations (i.e. prediction errors) are detected. In this way, prefrontal cortex contributes to reducing uncertainty in prediction systems. An emergent outcome of this homeostatic view may be the flexible and adaptive control that prefrontal cortex is known to implement (i.e. working memory) in the most challenging of situations. Compromise to any of the prediction circuits should result in rigid and suboptimal decision making and memory as seen in addiction and neurological disease. © 2013 The Authors. Hippocampus Published by Wiley Periodicals, Inc.

Working memory retrieval as a decision process

Working memory (WM) is a core cognitive process fundamental to human behavior, yet the mechanisms underlying it remain highly controversial. Here we provide a new framework for understanding retrieval of information from WM, conceptualizing it as a decision based on the quality of internal evidence. Recent findings have demonstrated that precision of WM decreases with memory load. If WM retrieval uses a decision process that depends on memory quality, systematic changes in response time distribution should occur as a function of WM precision. We asked participants to view sample arrays and, after a delay, report the direction of change in location or orientation of a probe. As WM precision deteriorated with increasing memory load, retrieval time increased systematically. Crucially, the shape of reaction time distributions was consistent with a linear accumulator decision process. Varying either task relevance of items or maintenance duration influenced memory precision, with corresponding shifts in retrieval time. These results provide strong support for a decision-making account of WM retrieval based on noisy storage of items. Furthermore, they show that encoding, maintenance, and retrieval in WM need not be considered as separate processes, but may instead be conceptually unified as operations on the same noise-limited, neural representation.

Real and implied motion at the center of gaze

Even though the dynamicity of our environment is a given, much of what we know on fixation selection comes from studies of static scene viewing. We performed a direct comparison of fixation selection on static and dynamic visual stimuli and investigated how far identical mechanisms drive these. We recorded eye movements while participants viewed movie clips of natural scenery and static frames taken from the same movies. Both were presented in the same high spatial resolution (1080 × 1920 pixels). The static condition allowed us to check whether local movement features computed from movies are salient even when presented as single frames. We observed that during the first second of viewing, movement and static features are equally salient in both conditions. Furthermore, predictability of fixations based on movement features decreased faster when viewing static frames as compared with viewing movie clips. Yet even during the later portion of static-frame viewing, the predictive value of movement features was still high above chance. Moreover, we demonstrated that, whereas the sets of movement and static features were statistically dependent within these sets, respectively, no dependence was observed between the two sets. Based on these results, we argue that implied motion is predictive of fixation similarly to real movement and that the onset of motion in natural stimuli is more salient than ongoing movement is. The present results allow us to address to what extent and when static image viewing is similar to the perception of a dynamic environment.

In pursuit of delay-related brain activity for anticipatory eye movements

How the brain stores motion information and subsequently uses it to follow a moving target is largely unknown. This is mainly due to previous fMRI studies using paradigms in which the eye movements cannot be segregated from the storage of this motion information. To avoid this problem we used a novel paradigm designed in our lab in which we interlaced a delay (2, 4 or 6 seconds) between the 1^st and 2^nd presentation of a moving stimulus. Using this design we could examine brain activity during a delay period using fMRI and have subsequently found a number of brain areas that reveal sustained activity during predictive pursuit. These areas include, the V5 complex and superior parietal lobe. This study provides new evidence for the network involved in the storage of visual information to generate early motor responses in pursuit.

A review of cell assemblies

Since the cell assembly (CA) was hypothesised, it has gained substantial support and is believed to be the neural basis of psychological concepts. A CA is a relatively small set of connected neurons, that through neural firing can sustain activation without stimulus from outside the CA, and is formed by learning. Extensive evidence from multiple single unit recording and other techniques provides support for the existence of CAs that have these properties, and that their neurons also spike with some degree of synchrony. Since the evidence is so broad and deep, the review concludes that CAs are all but certain. A model of CAs is introduced that is informal, but is broad enough to include, e.g. synfire chains, without including, e.g. holographic reduced representation. CAs are found in most cortical areas and in some sub-cortical areas, they are involved in psychological tasks including categorisation, short-term memory and long-term memory, and are central to other tasks including working memory. There is currently insufficient evidence to conclude that CAs are the neural basis of all concepts. A range of models have been used to simulate CA behaviour including associative memory and more process- oriented tasks such as natural language parsing. Questions involving CAs, e.g. memory persistence, CAs' complex interactions with brain waves and learning, remain unanswered. CA research involves a wide range of disciplines including biology and psychology, and this paper reviews literature directly related to the CA, providing a basis of discussion for this interdisciplinary community on this important topic. Hopefully, this discussion will lead to more formal and accurate models of CAs that are better linked to neuropsychological data.

Visual working memory contents bias ambiguous structure from motion perception

The way we perceive the visual world depends crucially on the state of the observer. In the present study we show that what we are holding in working memory (WM) can bias the way we perceive ambiguous structure from motion stimuli. Holding in memory the percept of an unambiguously rotating sphere influenced the perceived direction of motion of an ambiguously rotating sphere presented shortly thereafter. In particular, we found a systematic difference between congruent dominance periods where the perceived direction of the ambiguous stimulus corresponded to the direction of the unambiguous one and incongruent dominance periods. Congruent dominance periods were more frequent when participants memorized the speed of the unambiguous sphere for delayed discrimination than when they performed an immediate judgment on a change in its speed. The analysis of dominance time-course showed that a sustained tendency to perceive the same direction of motion as the prior stimulus emerged only in the WM condition, whereas in the attention condition perceptual dominance dropped to chance levels at the end of the trial. The results are explained in terms of a direct involvement of early visual areas in the active representation of visual motion in WM.

Visual short-term memory for global motion revealed by directional and speed-tuned masking

Neurobehavioral research with non-human primates has shown that different attributes of motion stimuli, such as direction and speed can be stored in visual short-term memory (VSTM) with a high degree of accuracy. We examined VSTM for global motion with a memory masking paradigm to determine which stimulus attributes are important in the storage process. We presented in two visual quadrants global motion random dot kinematograms (RDKs), whereas in the two remaining visual quadrants we presented random-motion RDKs. This pattern of stimulation was displayed in two distinct temporal intervals, i.e., sample and test stimuli (duration: 200 ms), separated in time by a 3.2-s delay period. During the delay period a random- or directional-motion mask was presented briefly (200 ms) either at the beginning, in the middle or at the end of the delay period. The results showed that the mask mainly interferes with performance when displayed 200 ms after the offset of the sample and when it had a coherent direction rather than random directions. Moreover, the mask is significantly more effective when its direction and speed matched that of the remembered sample. These results support the notion that the memory representation of global motion is selective for direction and speed, being compromised by intervening directional stimuli presented immediately after the encoding phase. Moreover, this selectivity suggests that the same neural mechanisms involved in the processing of global motion may be recruited for its storage.

Working memory and decision processes in visual area v4

Recognizing and responding to a remembered stimulus requires the coordination of perception, working memory, and decision-making. To investigate the role of visual cortex in these processes, we recorded responses of single V4 neurons during performance of a delayed match-to-sample task that incorporates rapid serial visual presentation of natural images. We found that neuronal activity during the delay period after the cue but before the images depends on the identity of the remembered image and that this change persists while distractors appear. This persistent response modulation has been identified as a diagnostic criterion for putative working memory signals; our data thus suggest that working memory may involve reactivation of sensory neurons. When the remembered image reappears in the neuron’s receptive field, visually evoked responses are enhanced; this match enhancement is a diagnostic criterion for decision. One model that predicts these data is the matched filter hypothesis, which holds that during search V4 neurons change their tuning so as to match the remembered cue, and thus become detectors for that image. More generally, these results suggest that V4 neurons participate in the perceptual, working memory, and decision processes that are needed to perform memory-guided decision-making.