Weak Evidence for Neural Correlates of Task-Switching in Macaque V1

A central goal of systems neuroscience is to understand how populations of sensory neurons encode and relay information to the rest of the brain. Three key quantities of interest are (i) how mean neural activity depends on the stimulus (sensitivity), (ii) how neural activity (co)varies around the mean (noise correlations), and (iii) how predictive these variations are of the subject's behavior (choice probability). Previous empirical work suggests that both choice probability and noise correlations are affected by task training, with decision-related information fed back to sensory areas and aligned to neural sensitivity on a task-by-task basis. We used Utah arrays to record activity from populations of V1 neurons from two macaque monkeys who were trained to switch between two coarse orientation-discrimination tasks. Surprisingly, we find no evidence for significant trial-by-trial changes in noise covariance between tasks, nor do we find a consistent relationship between neural sensitivity and choice probability, despite recording from well-tuned task-sensitive neurons, many of which were histologically confirmed to be in supragranular V1, and despite behavioral evidence that the monkeys switched their strategy between tasks. Thus our data at best provide weak support for the hypothesis that trial-by-trial task-switching induces changes to noise correlations and choice probabilities in V1. However, our data agrees with a recent finding of a single "choice axis" across tasks. It also raises the intriguing possibility that choice-related signals in early sensory areas are less indicative of task learning per se, and instead reflect perceptual learning that occurs in highly over-trained subjects.

Munc13 supports fusogenicity of non-docked vesicles at synapses with disrupted active zones

Active zones consist of protein scaffolds that are tightly attached to the presynaptic plasma membrane. They dock and prime synaptic vesicles, couple them to voltage-gated Ca²⁺ channels, and direct neurotransmitter release toward postsynaptic receptor domains. Simultaneous RIM + ELKS ablation disrupts these scaffolds, abolishes vesicle docking, and removes active zone-targeted Munc13, but some vesicles remain releasable. To assess whether this enduring vesicular fusogenicity is mediated by non-active zone-anchored Munc13 or is Munc13-independent, we ablated Munc13-1 and Munc13-2 in addition to RIM + ELKS in mouse hippocampal neurons. The hextuple knockout synapses lacked docked vesicles, but other ultrastructural features were near-normal despite the strong genetic manipulation. Removing Munc13 in addition to RIM + ELKS impaired action potential-evoked vesicle fusion more strongly than RIM + ELKS knockout by further decreasing the releasable vesicle pool. Hence, Munc13 can support some fusogenicity without RIM and ELKS, and presynaptic recruitment of Munc13, even without active zone anchoring, suffices to generate some fusion-competent vesicles.

Illusions, Delusions, and Your Backwards Bayesian Brain: A Biased Visual Perspective

The retinal image is insufficient for determining what is "out there," because many different real-world geometries could produce any given retinal image. Thus, the visual system must infer which external cause is most likely, given both the sensory data and prior knowledge that is either innate or learned via interactions with the environment. We will describe a general framework of "hierarchical Bayesian inference" that we and others have used to explore the role of cortico-cortical feedback in the visual system, and we will further argue that this approach to "seeing" makes our visual systems prone to perceptual errors in a variety of different ways. In this deliberately provocative and biased perspective, we argue that the neuromodulator, dopamine, may be a crucial link between neural circuits performing Bayesian inference and the perceptual idiosyncrasies of people with schizophrenia.

Gene Transfer with AAV9-PHP.B Rescues Hearing in a Mouse Model of Usher Syndrome 3A and Transduces Hair Cells in a Non-human Primate

Hereditary hearing loss often results from mutation of genes expressed by cochlear hair cells. Gene addition using AAV vectors has shown some efficacy in mouse models, but clinical application requires two additional advances. First, new AAV capsids must mediate efficient transgene expression in both inner and outer hair cells of the cochlea. Second, to have the best chance of clinical translation, these new vectors must also transduce hair cells in non-human primates. Here, we show that an AAV9 capsid variant, PHP.B, produces efficient transgene expression of a GFP reporter in both inner and outer hair cells of neonatal mice. We show also that AAV9-PHP.B mediates almost complete transduction of inner and outer HCs in a non-human primate. In a mouse model of Usher syndrome type 3A deafness (gene CLRN1), we use AAV9-PHP.B encoding Clrn1 to partially rescue hearing. Thus, we have identified a vector with promise for clinical treatment of hereditary hearing disorders, and we demonstrate, for the first time, viral transduction of the inner ear of a primate with an AAV vector.

Bottom-Up and Top-Down Input Augment the Variability of Cortical Neurons

Neurons in the cerebral cortex respond inconsistently to a repeated sensory stimulus, yet they underlie our stable sensory experiences. Although the nature of this variability is unknown, its ubiquity has encouraged the general view that each cell produces random spike patterns that noisily represent its response rate. In contrast, here we show that reversibly inactivating distant sources of either bottom-up or top-down input to cortical visual areas in the alert primate reduces both the spike train irregularity and the trial-to-trial variability of single neurons. A simple model in which a fraction of the pre-synaptic input is silenced can reproduce this reduction in variability, provided that there exist temporal correlations primarily within, but not between, excitatory and inhibitory input pools. A large component of the variability of cortical neurons may therefore arise from synchronous input produced by signals arriving from multiple sources.

A Modality-Specific Feedforward Component of Choice-Related Activity in MT

The activity of individual sensory neurons can be predictive of an animal's choices. These decision signals arise from network properties dependent on feedforward and feedback inputs; however, the relative contributions of these inputs are poorly understood. We determined the role of feedforward pathways to decision signals in MT by recording neuronal activity while monkeys performed motion and depth tasks. During each session, we reversibly inactivated V2 and V3, which provide feedforward input to MT that conveys more information about depth than motion. We thus monitored the choice-related activity of the same neuron both before and during V2/V3 inactivation. During inactivation, MT neurons became less predictive of decisions for the depth task but not the motion task, indicating that a feedforward pathway that gives rise to tuning preferences also contributes to decision signals. We show that our data are consistent with V2/V3 input conferring structured noise correlations onto the MT population.

Neuroanatomy goes viral!

The nervous system is complex not simply because of the enormous number of neurons it contains but by virtue of the specificity with which they are connected. Unraveling this specificity is the task of neuroanatomy. In this endeavor, neuroanatomists have traditionally exploited an impressive array of tools ranging from the Golgi method to electron microscopy. An ideal method for studying anatomy would label neurons that are interconnected, and, in addition, allow expression of foreign genes in these neurons. Fortuitously, nature has already partially developed such a method in the form of neurotropic viruses, which have evolved to deliver their genetic material between synaptically connected neurons while largely eluding glia and the immune system. While these characteristics make some of these viruses a threat to human health, simple modifications allow them to be used in controlled experimental settings, thus enabling neuroanatomists to trace multi-synaptic connections within and across brain regions. Wild-type neurotropic viruses, such as rabies and alpha-herpes virus, have already contributed greatly to our understanding of brain connectivity, and modern molecular techniques have enabled the construction of recombinant forms of these and other viruses. These newly engineered reagents are particularly useful, as they can target genetically defined populations of neurons, spread only one synapse to either inputs or outputs, and carry instructions by which the targeted neurons can be made to express exogenous proteins, such as calcium sensors or light-sensitive ion channels, that can be used to study neuronal function. In this review, we address these uniquely powerful features of the viruses already in the neuroanatomist's toolbox, as well as the aspects of their biology that currently limit their utility. Based on the latter, we consider strategies for improving viral tracing methods by reducing toxicity, improving control of transsynaptic spread, and extending the range of species that can be studied.

The quantitative methods boot camp: teaching quantitative thinking and computing skills to graduate students in the life sciences

The past decade has seen a rapid increase in the ability of biologists to collect large amounts of data. It is therefore vital that research biologists acquire the necessary skills during their training to visualize, analyze, and interpret such data. To begin to meet this need, we have developed a “boot camp” in quantitative methods for biology graduate students at Harvard Medical School. The goal of this short, intensive course is to enable students to use computational tools to visualize and analyze data, to strengthen their computational thinking skills, and to simulate and thus extend their intuition about the behavior of complex biological systems. The boot camp teaches basic programming using biological examples from statistics, image processing, and data analysis. This integrative approach to teaching programming and quantitative reasoning motivates students’ engagement by demonstrating the relevance of these skills to their work in life science laboratories. Students also have the opportunity to analyze their own data or explore a topic of interest in more detail. The class is taught with a mixture of short lectures, Socratic discussion, and in-class exercises. Students spend approximately 40% of their class time working through both short and long problems. A high instructor-to-student ratio allows students to get assistance or additional challenges when needed, thus enhancing the experience for students at all levels of mastery. Data collected from end-of-course surveys from the last five offerings of the course (between 2012 and 2014) show that students report high learning gains and feel that the course prepares them for solving quantitative and computational problems they will encounter in their research. We outline our course here which, together with the course materials freely available online under a Creative Commons License, should help to facilitate similar efforts by others.

Vesicular stomatitis virus enables gene transfer and transsynaptic tracing in a wide range of organisms

Current limitations in technology have prevented an extensive analysis of the connections among neurons, particularly within nonmammalian organisms. We developed a transsynaptic viral tracer originally for use in mice, and then tested its utility in a broader range of organisms. By engineering the vesicular stomatitis virus (VSV) to encode a fluorophore and either the rabies virus glycoprotein (RABV‐G) or its own glycoprotein (VSV‐G), we created viruses that can transsynaptically label neuronal circuits in either the retrograde or anterograde direction, respectively. The vectors were investigated for their utility as polysynaptic tracers of chicken and zebrafish visual pathways. They showed patterns of connectivity consistent with previously characterized visual system connections, and revealed several potentially novel connections. Further, these vectors were shown to infect neurons in several other vertebrates, including Old and New World monkeys, seahorses, axolotls, and Xenopus. They were also shown to infect two invertebrates, Drosophila melanogaster, and the box jellyfish, Tripedalia cystophora, a species previously intractable for gene transfer, although no clear evidence of transsynaptic spread was observed in these species. These vectors provide a starting point for transsynaptic tracing in most vertebrates, and are also excellent candidates for gene transfer in organisms that have been refractory to other methods. J. Comp. Neurol. 523:1639–1663, 2015. © 2015 Wiley Periodicals, Inc.

Feature attention for binocular disparity in primate area MT depends on tuning strength

Attending to a stimulus modulates the responses of sensory neurons that represent features of that stimulus, a phenomenon named "feature attention." For example, attending to a stimulus containing upward motion enhances the responses of upward-preferring direction-selective neurons in the middle temporal area (MT) and suppresses the responses of downward-preferring neurons, even when the attended stimulus is outside of the spatial receptive fields of the recorded neurons (Treue S, Martinez-Trujillo JC. Nature 399: 575-579, 1999). This modulation renders the representation of sensory information across a neuronal population more selective for the features present in the attended stimulus (Martinez-Trujillo JC, Treue S. Curr Biol 14: 744-751, 2004). We hypothesized that if feature attention modulates neurons according to their tuning preferences, it should also be sensitive to their tuning strength, which is the magnitude of the difference in responses to preferred and null stimuli. We measured how the effects of feature attention on MT neurons in rhesus monkeys (Macaca mulatta) depended on the relationship between features-in our case, direction of motion and binocular disparity-of the attended stimulus and a neuron's tuning for those features. We found that, as for direction, attention to stimuli containing binocular disparity cues modulated the responses of MT neurons and that the magnitude of the modulation depended on both a neuron's tuning preferences and its tuning strength. Our results suggest that modulation by feature attention may depend not just on which features a neuron represents but also on how well the neuron represents those features.

Cortical magnification plus cortical plasticity equals vision?

Most approaches to visual prostheses have focused on the retina, and for good reasons. The earlier that one introduces signals into the visual system, the more one can take advantage of its prodigious computational abilities. For methods that make use of microelectrodes to introduce electrical signals, however, the limited density and volume occupying nature of the electrodes place severe limits on the image resolution that can be provided to the brain. In this regard, non-retinal areas in general, and the primary visual cortex in particular, possess one large advantage: "magnification factor" (MF)-a value that represents the distance across a sheet of neurons that represents a given angle of the visual field. In the foveal representation of primate primary visual cortex, the MF is enormous-on the order of 15-20 mm/deg in monkeys and humans, whereas on the retina, the MF is limited by the optical design of the eye to around 0.3m m/deg. This means that, for an electrode array of a given density, a much higher-resolution image can be introduced into V1 than onto the retina (or any other visual structure). In addition to this tremendous advantage in resolution, visual cortex is plastic at many different levels ranging from a very local ability to learn to better detect electrical stimulation to higher levels of learning that permit human observers to adapt to radical changes to their visual inputs. We argue that the combination of the large magnification factor and the impressive ability of the cerebral cortex to learn to recognize arbitrary patterns, might outweigh the disadvantages of bypassing earlier processing stages and makes V1 a viable option for the restoration of vision.

Corticocortical feedback increases the spatial extent of normalization

Normalization has been proposed as a canonical computation operating across different brain regions, sensory modalities, and species. It provides a good phenomenological description of non-linear response properties in primary visual cortex (V1), including the contrast response function and surround suppression. Despite its widespread application throughout the visual system, the underlying neural mechanisms remain largely unknown. We recently observed that corticocortical feedback contributes to surround suppression in V1, raising the possibility that feedback acts through normalization. To test this idea, we characterized area summation and contrast response properties in V1 with and without feedback from V2 and V3 in alert macaques and applied a standard normalization model to the data. Area summation properties were well explained by a form of divisive normalization, which computes the ratio between a neuron's driving input and the spatially integrated activity of a "normalization pool." Feedback inactivation reduced surround suppression by shrinking the spatial extent of the normalization pool. This effect was independent of the gain modulation thought to mediate the influence of contrast on area summation, which remained intact during feedback inactivation. Contrast sensitivity within the receptive field center was also unaffected by feedback inactivation, providing further evidence that feedback participates in normalization independent of the circuit mechanisms involved in modulating contrast gain and saturation. These results suggest that corticocortical feedback contributes to surround suppression by increasing the visuotopic extent of normalization and, via this mechanism, feedback can play a critical role in contextual information processing.

Joint tuning for direction of motion and binocular disparity in macaque MT is largely separable

Neurons in sensory cortical areas are tuned to multiple dimensions, or features, of their sensory space. Understanding how single neurons represent multiple features is of great interest for determining the informative dimensions of the neurons' response, the decoding algorithms appropriate for extracting this information from the neuronal population, and for determining where specific transformations occur along the visual hierarchy. Despite the established role of cortical area MT in judgments of motion and depth, it is not known how individual neurons jointly encode the two dimensions. We investigated the joint tuning of individual MT neurons for two visual features: direction of motion and binocular disparity, an important depth cue. We found that a separable, multiplicative combination of tuning for the two features can account for more than 90% of the variance in the joint tuning function for over 91% of MT neurons. These results suggest 1) that each feature can be read out independently from MT by simply averaging across the population without regard to the other feature and 2) that the inseparable representations seen in subsequent areas, such as MST, must be computed beyond MT. Intriguingly, we found that the remaining nonseparable component of the joint tuning function often manifested as small but systematic changes in the neurons' preferences for one feature as the other one was varied. We believe this reflects the local columnar organization of tuning for direction and binocular disparity in MT, indicating that joint tuning may provide a new tool with which to probe functional architecture.

Segregation of feedforward and feedback projections in mouse visual cortex

Hierarchical organization is a common feature of mammalian neocortex. Neurons that send their axons from lower to higher areas of the hierarchy are referred to as "feedforward" (FF) neurons, whereas those projecting in the opposite direction are called "feedback" (FB) neurons. Anatomical, functional, and theoretical studies suggest that these different classes of projections play fundamentally different roles in perception. In primates, laminar differences in projection patterns often distinguish the two projection streams. In rodents, however, these differences are less clear, despite an established hierarchy of visual areas. Thus the rodent provides a strong test of the hypothesis that FF and FB neurons form distinct populations. We tested this hypothesis by injecting retrograde tracers into two different hierarchical levels of mouse visual cortex (area 17 and anterolateral area [AL]) and then determining the relative proportions of double-labeled FF and FB neurons in an area intermediate to them (lateromedial area [LM]). Despite finding singly labeled neurons densely intermingled with no laminar segregation, we found few double-labeled neurons (≈5% of each singly labeled population). We also examined the development of FF and FB connections. FF connections were present at the earliest timepoint we examined (postnatal day 2, P2), while FB connections were not detectable until P11. Our findings indicate that, even in cortices without laminar segregation of FF and FB neurons, the two projection systems are largely distinct at the neuronal level and also differ with respect to the timing of their axonal outgrowth.

The role of V1 surround suppression in MT motion integration

Neurons in the primate extrastriate cortex are highly selective for complex stimulus features such as faces, objects, and motion patterns. One explanation for this selectivity is that neurons in these areas carry out sophisticated computations on the outputs of lower-level areas such as primary visual cortex (V1), where neuronal selectivity is often modeled in terms of linear spatiotemporal filters. However, it has long been known that such simple V1 models are incomplete because they fail to capture important nonlinearities that can substantially alter neuronal selectivity for specific stimulus features. Thus a key step in understanding the function of higher cortical areas is the development of realistic models of their V1 inputs. We have addressed this issue by constructing a computational model of the V1 neurons that provide the strongest input to extrastriate cortical middle temporal (MT) area. We find that a modest elaboration to the standard model of V1 direction selectivity generates model neurons with strong end-stopping, a property that is also found in the V1 layers that provide input to MT. With this computational feature in place, the seemingly complex properties of MT neurons can be simulated by assuming that they perform a simple nonlinear summation of their inputs. The resulting model, which has a very small number of free parameters, can simulate many of the diverse properties of MT neurons. In particular, we simulate the invariance of MT tuning curves to the orientation and length of tilted bar stimuli, as well as the accompanying temporal dynamics. We also show how this property relates to the continuum from component to pattern selectivity observed when MT neurons are tested with plaids. Finally, we confirm several key predictions of the model by recording from MT neurons in the alert macaque monkey. Overall our results demonstrate that many of the seemingly complex computations carried out by high-level cortical neurons can in principle be understood by examining the properties of their inputs.

Integrating motion and depth via parallel pathways

Processing of visual information is both parallel and hierarchical, with each visual area richly interconnected with other visual areas. An example of the parallel architecture of the primate visual system is the existence of two principal pathways providing input to the middle temporal visual area (MT): namely, a direct projection from striate cortex (V1), and a set of indirect projections that also originate in V1 but then relay through V2 and V3. Here we have reversibly inactivated the indirect pathways while recording from MT neurons and measuring eye movements in alert monkeys, a procedure that has enabled us to assess whether the two different input pathways are redundant or whether they carry different kinds of information. We find that this inactivation causes a disproportionate degradation of binocular disparity tuning relative to direction tuning in MT neurons, suggesting that the indirect pathways are important in the recovery of depth in three-dimensional scenes.

Temporal evolution of 2-dimensional direction signals used to guide eye movements

The smooth pursuit system must integrate many local motion measurements into a coherent estimate of target velocity. Several laboratories have studied this integration process using eye movements elicited by targets, such as tilted bars, containing conflicts between local motion signals measured along contours [one dimensional (1D)] and those measured at the bar's endpoints, or terminators [two dimensional (2D)]. The general finding is that 1D signals dominate early responses, whereas later components of the behavior are determined by 2D signals. We studied the dynamics of the integration process in macaque monkeys by systematically varying the relative proportions of 1D and 2D signals and the retinal eccentricities at which they appeared. Predictably, longer bars produced greater and longer-lasting contour-induced deviations. The evolution of the 2D response occurred over a period of 50-400 ms, depending on the relative proportions of 1D and 2D signals. As contours were displaced from the fovea the deviation decreased but much less so for early (1st 40 ms) than for late (subsequent 40 ms) pursuit initiation. These bottom-up effects could be overcome to a limited extent by the top-down influence of predictability. Finally, we observed that when animals were free to track any part of the bar, they spontaneously made short-latency saccades to the terminators on most trials, especially when the bars were tilted. This suggests an increased saliency of moving terminators, particularly when discrepancies exist among local motion signals.

Spatiotemporal structure of nonlinear subunits in macaque visual cortex

The primate visual system is arranged hierarchically, starting from the retina and continuing through a series of extrastriate visual areas. Selectivity for motion is first found in individual neurons in the primate visual cortex (V1), in which many simple cells respond selectively to the direction and speed of moving stimuli. Beyond simple cells, most studies of direction selectivity have focused on either V1 complex cells or neurons in the middle temporal area (MT/V5). To understand how visual information is transferred along this pathway, we have studied all three types of neurons, using a reverse correlation procedure to obtain high spatial and temporal resolution maps of activity for different motion stimuli. Most complex and MT cells showed strong second-order interactions, indicating that they were tuned for particular displacements of an apparent motion stimulus. The spatiotemporal structure of these interactions showed a high degree of similarity between the populations of V1 complex cells and MT cells, in terms of the spatiotemporal limits and preferences for motion and their two-dimensional spatial structure. Much of the structure in the V1 and MT second-order kernels could be accounted for on the basis of the first-order responses of V1 simple cells, under the assumption of a Reichardt or motion-energy type of computation.

Structure and function of visual area MT

The small visual area known as MT or V5 has played a major role in our understanding of the primate cerebral cortex. This area has been historically important in the concept of cortical processing streams and the idea that different visual areas constitute highly specialized representations of visual information. MT has also proven to be a fertile culture dish--full of direction- and disparity-selective neurons--exploited by many labs to study the neural circuits underlying computations of motion and depth and to examine the relationship between neural activity and perception. Here we attempt a synthetic overview of the rich literature on MT with the goal of answering the question, What does MT do?

Contrast Dependence of Suppressive Influences in Cortical Area MT of Alert Macaque

Visual neurons are often characterized in terms of their tuning for various stimulus properties, such as shape, color, and velocity. Generally, these tuning curves are further modulated by the overall intensity of the stimulus, such that increasing the contrast increases the firing rate, up to some maximum. In this paper, we describe the tuning of neurons in the middle temporal area (MT or V5) of macaque visual cortex for moving stimuli of varying contrast. We find that, for some MT neurons, tuning curves for stimulus direction, speed, and size are shaped in part by suppressive influences that are present at high stimulus contrast but weak or nonexistent at low contrast. For most neurons, the suppression is direction-specific and strongest for large, slow-moving stimuli. The surprising consequence of this phenomenon is that some MT neurons actually fire more vigorously to a large low-contrast stimulus than to one of high contrast. These results are consistent with recent perceptual observations, as well as with information-theoretic models, which hypothesize that the visual system seeks to reduce redundancy at high contrast while maintaining sensitivity at low contrast.

Integration of Contour and Terminator Signals in Visual Area MT of Alert Macaque

The integration of visual information is a critical task that is performed by neurons in the extrastriate cortex of the primate brain. For motion signals, integration is complicated by the geometry of the visual world, which renders some velocity measurements ambiguous and others incorrect. The ambiguity arises because neurons in the early stages of visual processing have small receptive fields, which can only recover the component of motion perpendicular to the orientation of a contour (the aperture problem). Unambiguous motion signals are located at end points and corners, which are referred to as terminators. However, when an object moves behind an occluding surface, motion measurements made at the terminators formed by the intersection of the object and the occluder are generally not consistent with the direction of object motion. To study how cortical neurons integrate these different motion cues, we used variations on the classic "barber pole" stimulus and measured the responses of neurons in the middle temporal area (MT or V5) of extrastriate cortex of alert macaque monkeys. Our results show that MT neurons are more strongly influenced by the unambiguous motion signals generated by terminators than to the ambiguous signals generated by contours. Furthermore, these neurons respond better to terminators that are intrinsic to a moving object than to those that are accidents of occlusion. V1 neurons show similar response patterns to local cues (contours and terminators), but for large stimuli, they do not reflect the global motion direction computed by MT neurons. These observations are consistent with psychophysical findings that show that our perception of moving objects often depends on the motion of terminators.

Taking strategies to task

In this issue of Neuron, Uka and DeAngelis report a neural signature of the strategy that monkeys' brains developed to solve a visual discrimination task by training them on one version of the task and testing them on another. Extensive training on one version caused decision networks in the animals' brains to ignore certain classes of neurons whose signals would have been useful on the modified version of the task used to test them.

End-stopping and the aperture problem: two-dimensional motion signals in macaque V1

Our perception of fine visual detail relies on small receptive fields at early stages of visual processing. However, small receptive fields tend to confound the orientation and velocity of moving edges, leading to ambiguous or inaccurate motion measurements (the aperture problem). Thus, it is often assumed that neurons in primary visual cortex (V1) carry only ambiguous motion information. Here we show that a subpopulation of V1 neurons is capable of signaling motion direction in a manner that is independent of contour orientation. Specifically, end-stopped V1 neurons obtain accurate motion measurements by responding only to the endpoints of long contours, a strategy which renders them largely immune to the aperture problem. Furthermore, the time course of end-stopping is similar to the time course of motion integration by MT neurons. These results suggest that cortical neurons might represent object motion by responding selectively to two-dimensional discontinuities in the visual scene.

Two-dimensional substructure of stereo and motion interactions in macaque visual cortex

The analysis of object motion and stereoscopic depth are important tasks that are begun at early stages of the primate visual system. Using sparse white noise, we mapped the receptive field substructure of motion and disparity interactions in neurons in V1 and MT of alert monkeys. Interactions in both regions revealed subunits similar in structure to V1 simple cells. For both motion and stereo, the scale and shape of the receptive field substructure could be predicted from conventional tuning for bars or dot-field stimuli, indicating that the small-scale interactions were repeated across the receptive fields. We also found neurons in V1 and in MT that were tuned to combinations of spatial and temporal binocular disparities, suggesting a possible neural substrate for the perceptual Pulfrich phenomenon. Our observations constrain computational and developmental models of motion-stereo integration.

Integration of motion cues for the initiation of smooth pursuit eye movements

To clearly see a moving object, an observer must rotate his or her eyes with a velocity that matches that of the object. Such rotations are called smooth pursuit eye movements, and they depend critically on the ability of the primate brain to integrate information about object velocity from various local motion signals. When the local motion signals are in conflict, it is possible to use smooth pursuit eye movements as a continuous read-out of the motion integration process. This review discusses the results of recent behavioral experiments that have taken this approach, along with relevant neurophysiological and computational studies.

Dynamic properties of neurons in cortical area MT in alert and anaesthetized macaque monkeys

In order to see the world with high spatial acuity, an animal must sample the visual image with many detectors that restrict their analyses to extremely small regions of space. The visual cortex must then integrate the information from these localized receptive fields to obtain a more global picture of the surrounding environment. We studied this process in single neurons within the middle temporal visual area (MT) of macaques using stimuli that produced conflicting local and global information about stimulus motion. Neuronal responses in alert animals initially reflected predominantly the ambiguous local motion features, but gradually converged to an unambiguous global representation. When the same animals were anaesthetized, the integration of local motion signals was markedly impaired even though neuronal responses remained vigorous and directional tuning characteristics were intact. Our results suggest that anaesthesia preferentially affects the visual processing responsible for integrating local signals into a global visual representation.

Visual processing: parallel-er and parallel-er

The mammalian visual system processes many different aspects of the visual scene in separate, parallel channels. Recent experiments suggest that the visual cortex, like the retina, forms parallel circuits even at very fine spatial scales.

Two-dimensional substructure of MT receptive fields

Neurons at progressively higher levels of the visual system have progressively larger, more complicated receptive fields, presumably constructed from simpler antecedent receptive fields. To study this hierarchical organization, we used sparse white noise to map receptive-field substructure (second order Wiener-like kernels) in an extrastriate motion processing area (MT) of alert monkeys. The maps revealed a clear substructure, on a spatial scale comparable to the receptive fields of the V1 inputs. There were both facilitatory and suppressive interactions that differed in spatial organization and time course. Directional interactions were remarkably precise over a very small spatial range, and reversed when successive stimuli reversed contrast--a neural correlate of "reverse phi" motion perception. The maps of some cells had an unexpected, curved shape, which challenges existing models for direction selectivity.

Temporal dynamics of a neural solution to the aperture problem in visual area MT of macaque brain

A critical step in the interpretation of the visual world is the integration of the various local motion signals generated by moving objects. This process is complicated by the fact that local velocity measurements can differ depending on contour orientation and spatial position. Specifically, any local motion detector can measure only the component of motion perpendicular to a contour that extends beyond its field of view. This "aperture problem" is particularly relevant to direction-selective neurons early in the visual pathways, where small receptive fields permit only a limited view of a moving object. Here we show that neurons in the middle temporal visual area (known as MT or V5) of the macaque brain reveal a dynamic solution to the aperture problem. MT neurons initially respond primarily to the component of motion perpendicular to a contour's orientation, but over a period of approximately 60 ms the responses gradually shift to encode the true stimulus direction, regardless of orientation. We also report a behavioural correlate of these neural responses: the initial velocity of pursuit eye movements deviates in a direction perpendicular to local contour orientation, suggesting that the earliest neural responses influence the oculomotor response.

Center-surround interactions in the middle temporal visual area of the owl monkey

Microelectrode recording and 2-deoxyglucose (2dg) labeling were used to investigate center-surround interactions in the middle temporal visual area (MT) of the owl monkey. These techniques revealed columnar groups of neurons whose receptive fields had opposite types of center-surround interaction with respect to moving visual stimuli. In one type of column, neurons responded well to objects such as a single bar or spot but poorly to large textured stimuli such as random dots. This was often due to the fact that the receptive fields had antagonistic surrounds: surround motion in the same direction as that preferred by the center suppressed responses, thus rendering these neurons unresponsive to wide-field motion. In the second set of complementary, interdigitated columns, neuronal receptive fields had reinforcing surrounds and responded optimally to wide-field motion. This functional organization could not be accounted for by systematic differences in binocular disparity. Within both column types, neurons whose receptive fields exhibited center-surround interactions were found less frequently in the input layers compared with the other layers. Additional tests were done on single units to examine the nature of the center-surround interactions. The direction tuning of the surround was broader than that of the center, and the preferred direction, with respect to that of the center, tended to be either in the same or opposite direction and only rarely in orthogonal directions. Surround motion at various velocities modulated the overall responsiveness to centrally placed moving stimuli, but it did not produce shifts in the peaks of the center's tuning curves for either direction or speed. In layers 3B and 5 of the local motion processing columns, a number of neurons responded only to local motion contrast but did so over a region of the visual field that was much larger than the optimal stimulus size. The central feature of this receptive field type was the generalization of surround antagonism over retinotopic space-a property similar to other "complex" receptive fields described previously. The columnar organization of different types of center-surround interactions may reflect the initial segregation of visual motion information into wide-field and local motion contrast systems that serve complementary functions in visual motion processing. Such segregation appears to occur at later stages of the macaque motion processing stream, in the medial superior temporal area (MST), and has also been described in invertebrate visual systems where it appears to be involved in the important function of distinguishing background motion from object motion.

Segregation of object and background motion in visual area MT: effects of microstimulation on eye movements

To track a moving object, its motion must first be distinguished from that of the background. The center-surround properties of neurons in the middle temporal visual area (MT) may be important for signaling the relative motion between object and background. To test this, we microstimulated within MT and measured the effects on monkeys' eye movements to moving targets. We found that stimulation at "local motion" sites, where receptive fields possessed antagonistic surrounds, shifted pursuit in the preferred direction of the neurons, whereas stimulation at "wide-field motion" sites shifted pursuit in the opposite, or null, direction. We propose that activating wide-field sites simulated background motion, thus inducing a target motion signal in the opposite direction. Our results support the hypothesis that neuronal center-surround mechanisms contribute to the behavioral segregation of objects from the background.

Specificity of projections from wide-field and local motion-processing regions within the middle temporal visual area of the owl monkey

The middle temporal visual area (MT) of the owl monkey is anatomically organized with respect to both preferred direction of motion and different types of center-surround interaction. The latter organization consists of clusters of neurons whose receptive fields have antagonistic surrounds that render them unresponsive to wide-field motion (local motion columns) interdigitated with groups of neurons whose receptive fields have additive surrounds and thus respond best to wide-field motion (wide-field motion columns). To learn whether the information in these regions remained segregated further along the visual pathways, we made injections of retrograde tracers into two visual areas to which MT projects [the medial superior temporal area (MST) and fundus of the superior temporal sulcus (FST)] and then labeled the wide-field and local organization using 2-deoxyglucose. In complementary experiments, we injected anterograde tracers into regions of MT that we had mapped using microelectrode recordings. Injections into both dorsal FST and ventral MST labeled clusters of cell bodies in MT that were concentrated within wide-field motion columns, whereas injections into dorsal MST labeled neurons predominantly within local motion columns. Results from the anterograde tracer experiments corroborated these findings. The high degree of specificity in the connections reinforces a model of functional organization for wide-field versus local motion processing within MT. Our data support the previously reported division of FST into separate dorsal and ventral areas, and they also suggest that MST of the owl monkey is, like MST of the macaque, functionally organized with respect to local versus wide-field motion processing.

Maps of complex motion selectivity in the superior temporal cortex of the alert macaque monkey: a double-label 2-deoxyglucose study

The superior temporal sulcus (STS) of the macaque monkey contains multiple visual areas. Many neurons within these regions respond selectively to motion direction and to more complex motion patterns, such as expansion, contraction and rotation. Single-unit recording and optical recording studies in MT/MST suggest that cells with similar tuning properties are clustered into columns extending through multiple cortical layers. In this study, we used a double-label 2-deoxyglucose technique in awake, behaving macaque monkeys to clarify this functional organization. This technique allowed us to label, in a single animal, two populations of neurons responding to two different visual stimuli. In one monkey we compared expansion with contraction; in a second monkey we compared expansion with clockwise rotation. Within the STS we found a patchy arrangement of cortical columns with alternating stimulus selectivity: columns of neurons preferring expansion versus contraction were more widely separated than those selective for expansion versus rotation. This mosaic of interdigitating columns on the floor and posterior bank of the STS included area MT and some neighboring regions of cortex, perhaps including area MST.

How is a sensory map read Out? Effects of microstimulation in visual area MT on saccades and smooth pursuit eye movements

To generate behavioral responses based on sensory input, motor areas of the brain must interpret, or "read out," signals from sensory maps. Our experiments tested several algorithms for how the motor systems for smooth pursuit and saccadic eye movements might extract a usable signal of target velocity from the distributed representation of velocity in the middle temporal visual area (MT or V5). Using microstimulation, we attempted to manipulate the velocity information within MT while monkeys tracked a moving visual stimulus. We examined the effects of the microstimulation on smooth pursuit and on the compensation for target velocity shown by saccadic eye movements. Microstimulation could alter both the speed and direction of the motion estimates of both types of eye movements and could also cause monkeys to generate pursuit even when the visual target was actually stationary. The pattern of alterations suggests that microstimulation can introduce an additional velocity signal into MT and that the pursuit and saccadic systems usually compute a vector average of the visually evoked and microstimulation-induced velocity signals (pursuit, 55 of 122 experiments; saccades, 70 of 122). Microstimulation effects in a few experiments were consistent with vector summation of these two signals (pursuit, 6 of 122; saccades, 2 of 122). In the remainder of the experiments, microstimulation caused either an apparent impairment in motion processing (pursuit, 47 of 122; saccades, 41 of 122) or had no effect (pursuit, 14 of 122; saccades, 9 of 122). Within individual experiments, the effects on pursuit and saccades were usually similar, but the occasional striking exception suggests that the two eye movement systems may perform motion computations somewhat independently.

Functional analysis of human MT and related visual cortical areas using magnetic resonance imaging

Using noninvasive functional magnetic resonance imaging (fMRI) technique, we analyzed the responses in human area MT with regard to visual motion, color, and luminance contrast sensitivity, and retinotopy. As in previous PET studies, we found that area MT responded selectively to moving (compared to stationary) stimuli. The location of human MT in the present fMRI results is consistent with that of MT in earlier PET and anatomical studies. In addition we found that area MT has a much higher contrast sensitivity than that in several other areas, including primary visual cortex (V1). Functional MRI half-amplitudes in V1 and MT occurred at approximately 15% and 1% luminance contrast, respectively. High sensitivity to contrast and motion in MT have been closely associated with magnocellular stream specialization in nonhuman primates. Human psychophysics indicates that visual motion appears to diminish when moving color-varying stimuli are equated in luminance. Electrophysiological results from macaque MT suggest that the human percept could be due to decreases in firing of area MT cells at equiluminance. We show here that fMRI activity in human MT does in fact decrease at and near individually measured equiluminance. Tests with visuotopically restricted stimuli in each hemifield produced spatial variations in fMRI activity consistent with retinotopy in human homologs of macaque areas V1, V2, V3, and VP. Such activity in area MT appeared much less retinotopic, as in macaque. However, it was possible to measure the interhemispheric spread of fMRI activity in human MT (half amplitude activation across the vertical meridian = approximately 15 degrees).

Segregation of global and local motion processing in primate middle temporal visual area

The early stages of primate visual processing appear to be divided up into several component parts so that, for example, colour, form and motion are analysed by anatomically distinct streams. We have found that further subspecialization occurs within the motion processing stream. Neurons representing two different kinds of information about visual motion are segregated in columnar fashion within the middle temporal area of the owl monkey. These columns can be distinguished by labelling with 2-deoxyglucose in response to large-field random-dot patterns. Neurons in lightly labelled interbands have receptive fields with antagonistic surrounds: the response to a centrally placed moving stimulus is suppressed by motion in the surround. Neurons in more densely labelled bands have surrounds that reinforce the centre response so that they integrate motion cues over large areas of the visual field. Interband cells carry information about local motion contrast that may be used to detect motion boundaries or to indicate retinal slip during visual tracking. Band cells encode information about global motion that might be useful for orienting the animal in its environment.

Single-unit and 2-deoxyglucose studies of side inhibition in macaque striate cortex

In the course of studies to map spatial frequency tuning of neurons in layers 2 and 3 of macaque striate cortex, we found that a high proportion (70%) of cells in the interblob regions responded poorly to full-field gratings, compared with responses to single bars, edges, or delimited gratings. This was most often due to side inhibition, in which increasing the number of cycles of a grating placed within the cell's receptive field causes progressive inhibition of response. Quantitative receptive-field mappings showed, however, that the inhibition can occur within the region activated by a bar, as well as beyond it. The inhibition appears to be orientation-selective, in that a surround grating was more effective at inhibiting the response to a center grating patch if it was of similar orientation. 2-Deoxyglucose experiments confirmed that side inhibition is very widespread in the interblobs of layers 2 and 3 and suggested that it is reduced or lacking in layers 4A through 6. Since layers 2 and 3 of striate cortex are the major source of cortical projections to area V2 and beyond, the prevalence of side stopping in these laminae has implications for theories of cortical visual function. Side-stopped interblob cells may be acting as "contour-pass filters" that filter out redundant information in textured or noisy surfaces, focusing subsequent form processing on contrasts corresponding to object boundaries.

Spatial frequency tuning of single units in macaque supragranular striate cortex

2-Deoxyglucose experiments have raised the possibility of a functional organization for spatial frequency in macaque striate cortex. To analyze this possibility with better spatial resolution, we made tangential microelectrode penetrations at constant eccentricity through supragranular striate cortex in 7 anesthetized, paralyzed macaque monkeys. We recorded from 121 single units. The data fell into two distinct populations with respect to mean preferred spatial frequency: (i) interblob cells (3.8 +/- 2.0 cycles/degree, n = 83) and (ii) blob cells (1.1 +/- 0.8 cycles/degree, n = 38; P less than 0.001). Beyond this, we found no evidence for an orderly mapping of spatial frequency optima. At blob-interblob borders, we observed abrupt shifts from low, relatively uniform spatial frequency optima (blobs) to higher optima that varied unsystematically (interblobs). The spatial frequency optima (low vs. high) and nature of the tuning curves (low-pass vs. band-pass) in blob vs. interblob cells correlate well with psychophysical measures of the same differences for the chrominance vs. luminance channels. These data are consistent with a functional subdivision of striate cortex in which blob cells carry information concerned predominantly with color and interblob neurons carry information important for form analysis.