Integration and segregation of multiple motion signals by neurons in area MT of primate

The role of binocular disparity and attention in the neural representation of multiple moving stimuli in the visual cortex

Segmenting visual scenes into distinct objects and surfaces is a fundamental visual process, with stereoscopic depth and motion serving as crucial cues. However, how the visual system uses these cues to segment multiple objects is not fully understood. We investigated how neurons in the middle-temporal (MT) cortex of macaque monkeys represent overlapping surfaces at different depths, moving in different directions. Neuronal activity was recorded from three male monkeys during discrimination tasks under varying attention conditions. We found that neuronal responses to overlapping surfaces showed a robust bias toward the binocular disparity of one surface over the other. The disparity bias of a neuron was positively correlated with the neuron's disparity preference for a single surface. In two animals, neurons preferring near disparities of single surfaces (near neurons) showed a near bias for overlapping stimuli, while neurons preferring far disparities (far neurons) showed a far bias. In the third animal, both near and far neurons displayed a near bias, though the near neurons showed a stronger near bias. All three animals exhibited an initial near bias across neurons relative to the average of the responses to the individual surfaces. Although attention modulated neuronal responses, the disparity bias was not caused by attention. We also found that the effect of attention was consistent with object-based, rather than feature-based attention. We proposed a model in which the pool size of the neuron population that weighs the responses to individual stimulus components can be variable. This model is a novel extension of the standard normalization model and provides a unified explanation for the disparity bias across animals. Our results reveal how MT neurons encode multiple stimuli moving at different depths and present new evidence of response modulation by object-based attention. The disparity bias allows subgroups of neurons to preferentially represent individual surfaces of multiple stimuli at different depths, thereby facilitating segmentation.

Neural encoding of multiple motion speeds in visual cortical area MT

Segmenting objects from each other and their background is critical for vision. The speed at which objects move provides a salient cue for segmentation. However, how the visual system represents and differentiates multiple speeds is largely unknown. Here we investigated the neural encoding of multiple speeds of overlapping stimuli in the primate visual cortex. We first characterized the perceptual capacity of human and monkey subjects to segment spatially overlapping stimuli moving at different speeds. We then determined how neurons in the motion-sensitive, middle-temporal (MT) cortex of macaque monkeys encode multiple speeds. We made a novel finding that the responses of MT neurons to two speeds of overlapping stimuli showed a robust bias toward the faster speed component when both speeds were slow (≤ 20°/s). The faster-speed bias occurred even when a neuron had a slow preferred speed and responded more strongly to the slower component than the faster component when presented alone. The faster-speed bias emerged very early in neuronal response and was robust over time and to manipulations of motion direction and attention. As the stimulus speed increased, the faster-speed bias changed to response averaging. Our finding can be explained by a modified divisive normalization model, in which the weights for the speed components are proportional to the responses of a population of neurons elicited by the individual speeds. Our results suggest that the neuron population, referred to as the weighting pool, includes neurons that have a broad range of speed preferences. As a result, the response weights for the speed components are determined by the stimulus speeds and invariant to the speed preferences of individual neurons. Our findings help to define the neural encoding rule of multiple stimuli and provide new insight into the underlying neural mechanisms. The faster-speed bias would benefit behavioral tasks such as figure-ground segregation if figural objects tend to move faster than the background in the natural environment.

A neural correlate of perceptual segmentation in macaque middle temporal cortical area

High-resolution vision requires fine retinal sampling followed by integration to recover object properties. Importantly, accuracy is lost if local samples from different objects are intermixed. Thus, segmentation, grouping of image regions for separate processing, is crucial for perception. Previous work has used bi-stable plaid patterns, which can be perceived as either a single or multiple moving surfaces, to study this process. Here, we report a relationship between activity in a mid-level site in the primate visual pathways and segmentation judgments. Specifically, we find that direction selective middle temporal neurons are sensitive to texturing cues used to bias the perception of bi-stable plaids and exhibit a significant trial-by-trial correlation with subjective perception of a constant stimulus. This correlation is greater in units that signal global motion in patterns with multiple local orientations. Thus, we conclude the middle temporal area contains a signal for segmenting complex scenes into constituent objects and surfaces.

Behavioural and neural indices of perceptual decision-making in autistic children during visual motion tasks

Many studies report atypical responses to sensory information in autistic individuals, yet it is not clear which stages of processing are affected, with little consideration given to decision-making processes. We combined diffusion modelling with high-density EEG to identify which processing stages differ between 50 autistic and 50 typically developing children aged 6-14 years during two visual motion tasks. Our pre-registered hypotheses were that autistic children would show task-dependent differences in sensory evidence accumulation, alongside a more cautious decision-making style and longer non-decision time across tasks. We tested these hypotheses using hierarchical Bayesian diffusion models with a rigorous blind modelling approach, finding no conclusive evidence for our hypotheses. Using a data-driven method, we identified a response-locked centro-parietal component previously linked to the decision-making process. The build-up in this component did not consistently relate to evidence accumulation in autistic children. This suggests that the relationship between the EEG measure and diffusion-modelling is not straightforward in autistic children. Compared to a related study of children with dyslexia, motion processing differences appear less pronounced in autistic children. Exploratory analyses also suggest weak evidence that ADHD symptoms moderate perceptual decision-making in autistic children.

A neural index of inefficient evidence accumulation in dyslexia underlying slow perceptual decision making

Visual processing deficits have been widely reported in developmental dyslexia however the locus of cognitive dysfunction remains unclear. Here, we examined the neural correlates of perceptual decision-making using a dot-motion task and electroencephalography (EEG) and investigated whether presenting deficits were unique to children with dyslexia or if they were also evident in other, typically developing children with equally immature reading systems. Sixty-eight children participated: 32 with dyslexia (DD; 16 females); 21 age-matched controls (AM; 11 females) and 15 reading-matched controls (RM; 9 females). All participants completed a bilaterally presented random-dot-motion task while EEG was recorded. Neural signatures of low level sensory processing (steady state visual evoked potentials; SSVEPs), pre-target attentional bias (posterior α power), attentional orienting (N2), evidence accumulation (centro-parietal positive decision signal; CPP) and execution of a motor response (β) were obtained to dissect the temporal sequence of perceptual decision-making. Reading profile provided a score of relative lexical and sublexical skills for each participant. Although all groups performed comparably in terms of task accuracy and false alarm rate, the DD group were slower and demonstrated an earlier peak latency, reduced slope and lower amplitude of the CPP compared with both AM and RM controls. Reading profile was found to moderate the relationship between word reading ability, reaction time as well as CPP indices showing that lexical dyslexics responded more slowly and had a shallower slope, reduced amplitude and earlier latency of CPP waveforms than sublexical dyslexics. These findings suggest that children with dyslexia, particularly those with relatively poorer lexical abilities, have a reduced rate and peak of evidence accumulation as denoted by CPP markers yet remain slow in their overt response. This is in keeping with hypotheses that children with dyslexia have impairment in effectively sampling and processing evidence about visual motion stimuli.

Lévy walk dynamics explain gamma burst patterns in primate cerebral cortex

Lévy walks describe patterns of intermittent motion with variable step sizes. In complex biological systems, Lévy walks (non-Brownian, superdiffusive random walks) are associated with behaviors such as search patterns of animals foraging for food. Here we show that Lévy walks also describe patterns of oscillatory activity in primate cerebral cortex. We used a combination of empirical observation and modeling to investigate high-frequency (gamma band) local field potential activity in visual motion-processing cortical area MT of marmoset monkeys. We found that gamma activity is organized as localized burst patterns that propagate across the cortical surface with Lévy walk dynamics. Lévy walks are fundamentally different from either global synchronization, or regular propagating waves, because they include large steps that enable activity patterns to move rapidly over cortical modules. The presence of Lévy walk dynamics therefore represents a previously undiscovered mode of brain activity, and implies a novel way for the cortex to compute. We apply a biophysically realistic circuit model to explain that the Lévy walk dynamics arise from critical-state transitions between asynchronous and localized propagating wave states, and that these dynamics yield optimal spatial sampling of the cortical sheet. We hypothesise that Lévy walk dynamics could help the cortex to efficiently process variable inputs, and to find links in patterns of activity among sparsely spiking populations of neurons.

Cortico-Subcortical Functional Connectivity Profiles of Resting-State Networks in Marmosets and Humans

Understanding the similarity of cortico-subcortical networks topologies between humans and nonhuman primate species is critical to study the origin of network alternations underlying human neurologic and neuropsychiatric diseases. The New World common marmoset (Callithrix jacchus) has become popular as a nonhuman primate model for human brain function. Most marmoset connectomic research, however, has exclusively focused on cortical areas, with connectivity to subcortical networks less extensively explored. Here, we aimed to first isolate patterns of subcortical connectivity with cortical resting-state networks in awake marmosets using resting-state fMRI, then to compare these networks with those in humans using connectivity fingerprinting. In this study, we used 5 marmosets (4 males, 1 female). While we could match several marmoset and human resting-state networks based on their functional fingerprints, we also found a few striking differences, for example, strong functional connectivity of the default mode network with the superior colliculus in marmosets that was much weaker in humans. Together, these findings demonstrate that many of the core cortico-subcortical networks in humans are also present in marmosets, but that small, potentially functionally relevant differences exist.SIGNIFICANCE STATEMENT The common marmoset is becoming increasingly popular as an additional preclinical nonhuman primate model for human brain function. Here we compared the functional organization of cortico-subcortical networks in marmosets and humans using ultra-high field fMRI. We isolated the patterns of subcortical connectivity with cortical resting-state networks (RSNs) in awake marmosets using resting-state fMRI and then compared these networks with those in humans using connectivity fingerprinting. While we could match several marmoset and human RSNs based on their functional fingerprints, we also found several striking differences. Together, these findings demonstrate that many of the core cortico-subcortical RSNs in humans are also present in marmosets, but that small, potentially functionally relevant differences exist.

Spatial Arrangement Drastically Changes the Neural Representation of Multiple Visual Stimuli That Compete in More Than One Feature Domain

Natural scenes often contain multiple objects and surfaces. However, how neurons in the visual cortex represent multiple visual stimuli is not well understood. Previous studies have shown that, when multiple stimuli compete in one feature domain, the evoked neuronal response is biased toward the stimulus that has a stronger signal strength. We recorded from two male macaques to investigate how neurons in the middle temporal cortex (MT) represent multiple stimuli that compete in more than one feature domain. Visual stimuli were two random-dot patches moving in different directions. One stimulus had low luminance contrast and moved with high coherence, whereas the other had high contrast and moved with low coherence. We found that how MT neurons represent multiple stimuli depended on the spatial arrangement. When two stimuli were overlapping, MT responses were dominated by the stimulus component that had high contrast. When two stimuli were spatially separated within the receptive fields, the contrast dominance was abolished. We found the same results when using contrast to compete with motion speed. Our neural data and computer simulations using a V1-MT model suggest that the contrast dominance found with overlapping stimuli is due to normalization occurring at an input stage fed to MT, and MT neurons cannot overturn this bias based on their own feature selectivity. The interaction between spatially separated stimuli can largely be explained by normalization within MT. Our results revealed new rules on stimulus competition and highlighted the impact of hierarchical processing on representing multiple stimuli in the visual cortex.SIGNIFICANCE STATEMENT Previous studies have shown that the neural representation of multiple visual stimuli can be accounted for by a divisive normalization model. By using multiple stimuli that compete in more than one feature domain, we found that luminance contrast has a dominant effect in determining competition between multiple stimuli when they are overlapping but not spatially separated. Our results revealed that neuronal responses to multiple stimuli in a given cortical area cannot be simply predicted by the population neural responses elicited in that area by the individual stimulus components. To understand the neural representation of multiple stimuli, rather than considering response normalization only within the area of interest, one must consider the computations including normalization occurring along the hierarchical visual pathway.

Compound Stimuli Reveal the Structure of Visual Motion Selectivity in Macaque MT Neurons

Motion selectivity in primary visual cortex (V1) is approximately separable in orientation, spatial frequency, and temporal frequency (“frequency-separable”). Models for area MT neurons posit that their selectivity arises by combining direction-selective V1 afferents whose tuning is organized around a tilted plane in the frequency domain, specifying a particular direction and speed (“velocity-separable”). This construction explains “pattern direction-selective” MT neurons, which are velocity-selective but relatively invariant to spatial structure, including spatial frequency, texture and shape. We designed a set of experiments to distinguish frequency-separable and velocity-separable models and executed them with single-unit recordings in macaque V1 and MT. Surprisingly, when tested with single drifting gratings, most MT neurons’ responses are fit equally well by models with either form of separability. However, responses to plaids (sums of two moving gratings) tend to be better described as velocity-separable, especially for pattern neurons. We conclude that direction selectivity in MT is primarily computed by summing V1 afferents, but pattern-invariant velocity tuning for complex stimuli may arise from local, recurrent interactions.

Detection and analysis of spatiotemporal patterns in brain activity

There is growing evidence that population-level brain activity is often organized into propagating waves that are structured in both space and time. Such spatiotemporal patterns have been linked to brain function and observed across multiple recording methodologies and scales. The ability to detect and analyze these patterns is thus essential for understanding the working mechanisms of neural circuits. Here we present a mathematical and computational framework for the identification and analysis of multiple classes of wave patterns in neural population-level recordings. By drawing a conceptual link between spatiotemporal patterns found in the brain and coherent structures such as vortices found in turbulent flows, we introduce velocity vector fields to characterize neural population activity. These vector fields are calculated for both phase and amplitude of oscillatory neural signals by adapting optical flow estimation methods from the field of computer vision. Based on these velocity vector fields, we then introduce order parameters and critical point analysis to detect and characterize a diverse range of propagating wave patterns, including planar waves, sources, sinks, spiral waves, and saddle patterns. We also introduce a novel vector field decomposition method that extracts the dominant spatiotemporal structures in a recording. This enables neural data to be represented by the activity of a small number of independent spatiotemporal modes, providing an alternative to existing dimensionality reduction techniques which separate space and time components. We demonstrate the capabilities of the framework and toolbox with simulated data, local field potentials from marmoset visual cortex and optical voltage recordings from whole mouse cortex, and we show that pattern dynamics are non-random and are modulated by the presence of visual stimuli. These methods are implemented in a MATLAB toolbox, which is freely available under an open-source licensing agreement.

Structured activity such as propagating wave patterns at the level of neural circuits can arise from highly variable firing activity of individual neurons. This property makes the brain, a quintessential example of a complex system, analogous to other complex physical systems such as turbulent fluids, in which structured patterns like vortices similarly emerge from molecules that behave irregularly. In this study, by uniquely adapting techniques for the identification of coherent structures in fluid turbulence, we develop new analytical and computational methods for the reliable detection of a diverse range of propagating wave patterns in large-scale neural recordings, for comprehensive analysis and visualization of these patterns, and for analysis of their dominant spatiotemporal modes. We demonstrate that these methods can be used to uncover the essential spatiotemporal properties of neural population activity recorded by different modalities, thus offering new insights into understanding the working mechanisms of neural systems.

Intact perception of coherent motion, dynamic rigid form, and biological motion in chronic schizophrenia

BACKGROUND

Prior studies have documented biological motion perception deficits in schizophrenia, but it remains unclear whether the impairments arise from poor social cognition, perceptual organization, basic motion processing, or sustained attention/motivation. To address the issue, we had 24 chronic schizophrenia patients and 27 healthy controls perform three tasks: coherent motion, where subjects indicated whether a cloud of dots drifted leftward or rightward; dynamic rigid form, where subjects determined the tilt direction of a translating, point-light rectangle; and biological motion, where subjects judged whether a human point-light figure walked leftward or rightward. Task difficulty was staircase controlled and depended on the directional variability of the background dot motion. Catch trials were added to verify task attentiveness and engagement.

RESULTS

Patients and controls demonstrated similar performance thresholds and near-ceiling catch trial accuracy for each task (uncorrected ps > 0.1; ds < 0.35). In all but the coherent motion task, higher IQ correlated with better performance (ps < 0.001).

CONCLUSION

Schizophrenia patients have intact perception of motion coherence, dynamic rigid form, and biological motion at least for our sample and set-up. We speculate that previously documented biological motion perception deficits arose from task or stimulus differences or from group differences in IQ, attention, or motivation.

Criterion-free measurement of motion transparency perception at different speeds

Transparency perception often occurs when objects within the visual scene partially occlude each other or move at the same time, at different velocities across the same spatial region. Although transparent motion perception has been extensively studied, we still do not understand how the distribution of velocities within a visual scene contribute to transparent perception. Here we use a novel psychophysical procedure to characterize the distribution of velocities in a scene that give rise to transparent motion perception. To prevent participants from adopting a subjective decision criterion when discriminating transparent motion, we used an “odd-one-out,” three-alternative forced-choice procedure. Two intervals contained the standard—a random-dot-kinematogram with dot speeds or directions sampled from a uniform distribution. The other interval contained the comparison—speeds or directions sampled from a distribution with the same range as the standard, but with a notch of different widths removed. Our results suggest that transparent motion perception is driven primarily by relatively slow speeds, and does not emerge when only very fast speeds are present within a visual scene. Transparent perception of moving surfaces is modulated by stimulus-based characteristics, such as the separation between the means of the overlapping distributions or the range of speeds presented within an image. Our work illustrates the utility of using objective, forced-choice methods to reveal the mechanisms underlying motion transparency perception.

Development of stereotaxic recording system for awake marmosets (Callithrix jacchus)

The common marmoset has been proposed as a potential alternative to macaque monkey as a primate model for neuroscience and medical research. Here, we have newly developed a stereotaxic neuronal recording system for awake marmosets under the head-fixed condition by modifying that for macaque monkeys. Using this system, we recorded neuronal activity in the cerebral cortex of awake marmosets and successfully identified the primary motor cortex by intracortical microstimulation. Neuronal activities of deep brain structures, such as the basal ganglia, thalamus, and cerebellum, in awake marmosets were also successfully recorded referring to magnetic resonance images. Our system is suitable for functional mapping of the brain, since the large recording chamber allows access to arbitrary regions over almost the entire brain, and the recording electrode can be easily moved stereotaxically from one site to another. In addition, our system is desirable for neuronal recording during task performance to assess motor skills and cognitive function, as the marmoset sits in the marmoset chair and can freely use its hands. Moreover, our system can be used in combination with cutting-edge techniques, such as two-photon imaging and optogenetic manipulation. This recording system will contribute to boosting neuroscience and medical research using marmosets.

Directional Limits on Motion Transparency Assessed Through Colour-Motion Binding

Motion-defined transparency is the perception of two or more distinct moving surfaces at the same retinal location. We explored the limits of motion transparency using superimposed surfaces of randomly positioned dots defined by differences in motion direction and colour. In one experiment, dots were red or green and we varied the proportion of dots of a single colour that moved in a single direction ('colour-motion coherence') and measured the threshold direction difference for discriminating between two directions. When colour-motion coherences were high (e.g., 90% of red dots moving in one direction), a smaller direction difference was required to correctly bind colour with direction than at low coherences. In another experiment, we varied the direction difference between the surfaces and measured the threshold colour-motion coherence required to discriminate between them. Generally, colour-motion coherence thresholds decreased with increasing direction differences, stabilising at direction differences around 45°. Different stimulus durations were compared, and thresholds were higher at the shortest (150 ms) compared with the longest (1,000 ms) duration. These results highlight different yet interrelated aspects of the task and the fundamental limits of the mechanisms involved: the resolution of narrowly separated directions in motion processing and the local sampling of dot colours from each surface.

A Unifying Motif for Spatial and Directional Surround Suppression

In the visual system, the response to a stimulus in a neuron's receptive field can be modulated by stimulus context, and the strength of these contextual influences vary with stimulus intensity. Recent work has shown how a theoretical model, the stabilized supralinear network (SSN), can account for such modulatory influences, using a small set of computational mechanisms. Although the predictions of the SSN have been confirmed in primary visual cortex (V1), its computational principles apply with equal validity to any cortical structure. We have therefore tested the generality of the SSN by examining modulatory influences in the middle temporal area (MT) of the macaque visual cortex, using electrophysiological recordings and pharmacological manipulations. We developed a novel stimulus that can be adjusted parametrically to be larger or smaller in the space of all possible motion directions. We found, as predicted by the SSN, that MT neurons integrate across motion directions for low-contrast stimuli, but that they exhibit suppression by the same stimuli when they are high in contrast. These results are analogous to those found in visual cortex when stimulus size is varied in the space domain. We further tested the mechanisms of inhibition using pharmacological manipulations of inhibitory efficacy. As predicted by the SSN, local manipulation of inhibitory strength altered firing rates, but did not change the strength of surround suppression. These results are consistent with the idea that the SSN can account for modulatory influences along different stimulus dimensions and in different cortical areas.SIGNIFICANCE STATEMENT Visual neurons are selective for specific stimulus features in a region of visual space known as the receptive field, but can be modulated by stimuli outside of the receptive field. The SSN model has been proposed to account for these and other modulatory influences, and tested in V1. As this model is not specific to any particular stimulus feature or brain region, we wondered whether similar modulatory influences might be observed for other stimulus dimensions and other regions. We tested for specific patterns of modulatory influences in the domain of motion direction, using electrophysiological recordings from MT. Our data confirm the predictions of the SSN in MT, suggesting that the SSN computations might be a generic feature of sensory cortex.

Dissociation of Self-Motion and Object Motion by Linear Population Decoding That Approximates Marginalization

We use visual image motion to judge the movement of objects, as well as our own movements through the environment. Generally, image motion components caused by object motion and self-motion are confounded in the retinal image. Thus, to estimate heading, the brain would ideally marginalize out the effects of object motion (or vice versa), but little is known about how this is accomplished neurally. Behavioral studies suggest that vestibular signals play a role in dissociating object motion and self-motion, and recent computational work suggests that a linear decoder can approximate marginalization by taking advantage of diverse multisensory representations. By measuring responses of MSTd neurons in two male rhesus monkeys and by applying a recently-developed method to approximate marginalization by linear population decoding, we tested the hypothesis that vestibular signals help to dissociate self-motion and object motion. We show that vestibular signals stabilize tuning for heading in neurons with congruent visual and vestibular heading preferences, whereas they stabilize tuning for object motion in neurons with discrepant preferences. Thus, vestibular signals enhance the separability of joint tuning for object motion and self-motion. We further show that a linear decoder, designed to approximate marginalization, allows the population to represent either self-motion or object motion with good accuracy. Decoder weights are broadly consistent with a readout strategy, suggested by recent computational work, in which responses are decoded according to the vestibular preferences of multisensory neurons. These results demonstrate, at both single neuron and population levels, that vestibular signals help to dissociate self-motion and object motion.SIGNIFICANCE STATEMENT The brain often needs to estimate one property of a changing environment while ignoring others. This can be difficult because multiple properties of the environment may be confounded in sensory signals. The brain can solve this problem by marginalizing over irrelevant properties to estimate the property-of-interest. We explore this problem in the context of self-motion and object motion, which are inherently confounded in the retinal image. We examine how diversity in a population of multisensory neurons may be exploited to decode self-motion and object motion from the population activity of neurons in macaque area MSTd.

Visual Motion Discrimination by Propagating Patterns in Primate Cerebral Cortex

Visual stimuli can evoke waves of neural activity that propagate across the surface of visual cortical areas. The relevance of these waves for visual processing is unknown. Here, we measured the phase and amplitude of local field potentials (LFPs) in electrode array recordings from the motion-processing medial temporal (MT) area of anesthetized male marmosets. Animals viewed grating or dot-field stimuli drifting in different directions. We found that, on individual trials, the direction of LFP wave propagation is sensitive to the direction of stimulus motion. Propagating LFP patterns are also detectable in trial-averaged activity, but the trial-averaged patterns exhibit different dynamics and behaviors from those in single trials and are similar across motion directions. We show that this difference arises because stimulus-sensitive propagating patterns are present in the phase of single-trial oscillations, whereas the trial-averaged signal is dominated by additive amplitude effects. Our results demonstrate that propagating LFP patterns can represent sensory inputs at timescales relevant to visually guided behaviors and raise the possibility that propagating activity patterns serve neural information processing in area MT and other cortical areas.SIGNIFICANCE STATEMENT Propagating wave patterns are widely observed in the cortex, but their functional relevance remains unknown. We show here that visual stimuli generate propagating wave patterns in local field potentials (LFPs) in a movement-sensitive area of the primate cortex and that the propagation direction of these patterns is sensitive to stimulus motion direction. We also show that averaging LFP signals across multiple stimulus presentations (trial averaging) yields propagating patterns that capture different dynamic properties of the LFP response and show negligible direction sensitivity. Our results demonstrate that sensory stimuli can modulate propagating wave patterns reliably in the cortex. The relevant dynamics are normally masked by trial averaging, which is a conventional step in LFP signal processing.

Recurrent network dynamics reconciles visual motion segmentation and integration

In sensory systems, a range of computational rules are presumed to be implemented by neuronal subpopulations with different tuning functions. For instance, in primate cortical area MT, different classes of direction-selective cells have been identified and related either to motion integration, segmentation or transparency. Still, how such different tuning properties are constructed is unclear. The dominant theoretical viewpoint based on a linear-nonlinear feed-forward cascade does not account for their complex temporal dynamics and their versatility when facing different input statistics. Here, we demonstrate that a recurrent network model of visual motion processing can reconcile these different properties. Using a ring network, we show how excitatory and inhibitory interactions can implement different computational rules such as vector averaging, winner-take-all or superposition. The model also captures ordered temporal transitions between these behaviors. In particular, depending on the inhibition regime the network can switch from motion integration to segmentation, thus being able to compute either a single pattern motion or to superpose multiple inputs as in motion transparency. We thus demonstrate that recurrent architectures can adaptively give rise to different cortical computational regimes depending upon the input statistics, from sensory flow integration to segmentation.

Dynamic population codes of multiplexed stimulus features in primate area MT

The middle-temporal area (MT) of primate visual cortex is critical in the analysis of visual motion. Single-unit studies suggest that the response dynamics of neurons within area MT depend on stimulus features, but how these dynamics emerge at the population level, and how feature representations interact, is not clear. Here, we used multivariate classification analysis to study how stimulus features are represented in the spiking activity of populations of neurons in area MT of marmoset monkey. Using representational similarity analysis we distinguished the emerging representations of moving grating and dot field stimuli. We show that representations of stimulus orientation, spatial frequency, and speed are evident near the onset of the population response, while the representation of stimulus direction is slower to emerge and sustained throughout the stimulus-evoked response. We further found a spatiotemporal asymmetry in the emergence of direction representations. Representations for high spatial frequencies and low temporal frequencies are initially orientation dependent, while those for high temporal frequencies and low spatial frequencies are more sensitive to motion direction. Our analyses reveal a complex interplay of feature representations in area MT population response that may explain the stimulus-dependent dynamics of motion vision.NEW & NOTEWORTHY Simultaneous multielectrode recordings can measure population-level codes that previously were only inferred from single-electrode recordings. However, many multielectrode recordings are analyzed using univariate single-electrode analysis approaches, which fail to fully utilize the population-level information. Here, we overcome these limitations by applying multivariate pattern classification analysis and representational similarity analysis to large-scale recordings from middle-temporal area (MT) in marmoset monkeys. Our analyses reveal a dynamic interplay of feature representations in area MT population response.

Interpreting the dimensions of neural feature representations revealed by dimensionality reduction

Recent progress in understanding the structure of neural representations in the cerebral cortex has centred around the application of multivariate classification analyses to measurements of brain activity. These analyses have proved a sensitive test of whether given brain regions provide information about specific perceptual or cognitive processes. An exciting extension of this approach is to infer the structure of this information, thereby drawing conclusions about the underlying neural representational space. These approaches rely on exploratory data-driven dimensionality reduction to extract the natural dimensions of neural spaces, including natural visual object and scene representations, semantic and conceptual knowledge, and working memory. However, the efficacy of these exploratory methods is unknown, because they have only been applied to representations in brain areas for which we have little or no secondary knowledge. One of the best-understood areas of the cerebral cortex is area MT of primate visual cortex, which is known to be important in motion analysis. To assess the effectiveness of dimensionality reduction for recovering neural representational space we applied several dimensionality reduction methods to multielectrode measurements of spiking activity obtained from area MT of marmoset monkeys, made while systematically varying the motion direction and speed of moving stimuli. Despite robust tuning at individual electrodes, and high classifier performance, dimensionality reduction rarely revealed dimensions for direction and speed. We use this example to illustrate important limitations of these analyses, and suggest a framework for how to best apply such methods to data where the structure of the neural representation is unknown.

Spectral Signatures of Feedforward and Recurrent Circuitry in Monkey Area MT

Recordings of local field potential (LFP) in the visual cortex can show rhythmic activity at gamma frequencies (30-100 Hz). While the gamma rhythms in the primary visual cortex have been well studied, the structural and functional characteristics of gamma rhythms in extrastriate visual cortex are less clear. Here, we studied the spatial distribution and functional specificity of gamma rhythms in extrastriate middle temporal (MT) area of visual cortex in marmoset monkeys. We found that moving gratings induced narrowband gamma rhythms across cortical layers that were coherent across much of area MT. Moving dot fields instead induced a broadband increase in LFP in middle and upper layers, with weaker narrowband gamma rhythms in deeper layers. The stimulus dependence of LFP response in middle and upper layers of area MT appears to reflect the presence (gratings) or absence (dot fields and other textures) of strongly oriented contours. Our results suggest that gamma rhythms in these layers are propagated from earlier visual cortex, while those in the deeper layers may emerge in area MT.

Probing the Characteristics of Colour-Motion Binding and Its Dependence on Persistent Surface Segregation

Identifying the spatial and temporal characteristics of visual feature binding is a remaining challenge in the science of perception. Within the feature-binding literature, disparate findings have suggested the existence of more than one feature-binding mechanism with differing temporal resolutions. For example, one surprising result is that temporal alternations between two different feature pairings of colour and motion (e.g., orange dots moving left with blue dots moving right) support accurate conjunction discrimination at alternation frequencies of around 10 Hz and greater. However, at lower alternation frequencies around 5 Hz, conjunction discrimination falls to chance. To further investigate this effect, we present two experiments that probe the stimulus characteristics that facilitate or impede feature binding. Using novel manipulations of random dot kinematograms, we identify that facilitating surface representations through temporal integration can enable accurate conjunction discrimination at both intermediate and high alternation frequencies. We also offer a neurally plausible evidence accumulator model to describe these results, removing the need to suggest multiple binding mechanisms acting at different timescales. In effect, we propose a single, flexible binding process, whereby the relatively low temporal resolution for binding features can be circumvented by extracting them from rapidly formed and persistent surface representations.

Anatomical and functional neuroimaging in awake, behaving marmosets

The common marmoset (Callithrix jacchus) is a small New World monkey that has gained significant recent interest in neuroscience research, not only because of its compatibility with gene editing techniques, but also due to its tremendous versatility as an experimental animal model. Neuroimaging modalities, including anatomical (MRI) and functional magnetic resonance imaging (fMRI), complemented by two-photon laser scanning microscopy and electrophysiology, have been at the forefront of unraveling the anatomical and functional organization of the marmoset brain. High-resolution anatomical MRI of the marmoset brain can be obtained with remarkable cytoarchitectonic detail. Functional MRI of the marmoset brain has been used to study various sensory systems, including somatosensory, auditory, and visual pathways, while resting-state fMRI studies have unraveled functional brain networks that bear great correspondence to those previously described in humans. Two-photon laser scanning microscopy of the marmoset brain has enabled the simultaneous recording of neuronal activity from thousands of neurons with single cell spatial resolution. In this article, we aim to review the main results obtained by our group and by our colleagues in applying neuroimaging techniques to study the marmoset brain. © 2016 Wiley Periodicals, Inc. Develop Neurobiol 77: 373-389, 2017.

Design and implementation of embedded 8-channel receive-only arrays for whole-brain MRI and fMRI of conscious awake marmosets

PURPOSE

The common marmoset (Callithrix jacchus) is a New World primate of increasing interest to neuroscience and in translational brain research. The present work describes the design and implementation of individualized 8-channel receive-only radiofrequency (RF) coil arrays that provide whole-brain coverage and allow anatomical and functional MRI experiments in conscious, awake marmosets.

METHODS

The coil arrays were designed with their elements embedded inside individualized restraint helmets. The size, geometry, and arrangement of the coil elements were optimized to allow whole-brain coverage. Coil-to-coil decoupling was achieved by a combination of geometric decoupling and low input impedance preamplifiers. The performance of the embedded arrays was compared against that of one 8-channel receive-only array built to fit the external surface of the helmets.

RESULTS

Three individualized helmets with embedded coil arrays were built for three marmosets. Whole-brain coverage was achieved with high sensitivity extending over the entire cortex. Visual stimulation of conscious awake marmosets elicited robust BOLD fMRI responses in both primary and higher order visual areas of the occipitotemporal cortex.

CONCLUSION

The high sensitivity provided by embedded receive-only coil arrays allows both anatomical and functional MRI data to be obtained with high spatial resolution in conscious, awake marmosets. Magn Reson Med 78:387-398, 2017. © 2016 International Society for Magnetic Resonance in Medicine.

Discrepancies between Multi-Electrode LFP and CSD Phase-Patterns: A Forward Modeling Study

Multi-electrode recordings of local field potentials (LFPs) provide the opportunity to investigate the spatiotemporal organization of neural activity on the scale of several millimeters. In particular, the phases of oscillatory LFPs allow studying the coordination of neural oscillations in time and space and to tie it to cognitive processing. Given the computational roles of LFP phases, it is important to know how they relate to the phases of the underlying current source densities (CSDs) that generate them. Although CSDs and LFPs are distinct physical quantities, they are often (implicitly) identified when interpreting experimental observations. That this identification is problematic is clear from the fact that LFP phases change when switching to different electrode montages, while the underlying CSD phases remain unchanged. In this study we use a volume-conductor model to characterize discrepancies between LFP and CSD phase-patterns, to identify the contributing factors, and to assess the effect of different electrode montages. Although we focus on cortical LFPs recorded with two-dimensional (Utah) arrays, our findings are also relevant for other electrode configurations. We found that the main factors that determine the discrepancy between CSD and LFP phase-patterns are the frequency of the neural oscillations and the extent to which the laminar CSD profile is balanced. Furthermore, the presence of laminar phase-differences in cortical oscillations, as commonly observed in experiments, precludes identifying LFP phases with those of the CSD oscillations at a given cortical depth. This observation potentially complicates the interpretation of spike-LFP coherence and spike-triggered LFP averages. With respect to reference strategies, we found that the average-reference montage leads to larger discrepancies between LFP and CSD phases as compared with the referential montage, while the Laplacian montage reduces these discrepancies. We therefore advice to conduct analysis of two-dimensional LFP recordings using the Laplacian montage.

Correlations and Neuronal Population Information

Brain function involves the activity of neuronal populations. Much recent effort has been devoted to measuring the activity of neuronal populations in different parts of the brain under various experimental conditions. Population activity patterns contain rich structure, yet many studies have focused on measuring pairwise relationships between members of a larger population-termed noise correlations. Here we review recent progress in understanding how these correlations affect population information, how information should be quantified, and what mechanisms may give rise to correlations. As population coding theory has improved, it has made clear that some forms of correlation are more important for information than others. We argue that this is a critical lesson for those interested in neuronal population responses more generally: Descriptions of population responses should be motivated by and linked to well-specified function. Within this context, we offer suggestions of where current theoretical frameworks fall short.

Distributed and Dynamic Neural Encoding of Multiple Motion Directions of Transparently Moving Stimuli in Cortical Area MT

Segmenting visual scenes into distinct objects and surfaces is a fundamental visual function. To better understand the underlying neural mechanism, we investigated how neurons in the middle temporal cortex (MT) of macaque monkeys represent overlapping random-dot stimuli moving transparently in slightly different directions. It has been shown that the neuronal response elicited by two stimuli approximately follows the average of the responses elicited by the constituent stimulus components presented alone. In this scheme of response pooling, the ability to segment two simultaneously presented motion directions is limited by the width of the tuning curve to motion in a single direction. We found that, although the population-averaged neuronal tuning showed response averaging, subgroups of neurons showed distinct patterns of response tuning and were capable of representing component directions that were separated by a small angle--less than the tuning width to unidirectional stimuli. One group of neurons preferentially represented the component direction at a specific side of the bidirectional stimuli, weighting one stimulus component more strongly than the other. Another group of neurons pooled the component responses nonlinearly and showed two separate peaks in their tuning curves even when the average of the component responses was unimodal. We also show for the first time that the direction tuning of MT neurons evolved from initially representing the vector-averaged direction of slightly different stimuli to gradually representing the component directions. Our results reveal important neural processes underlying image segmentation and suggest that information about slightly different stimulus components is computed dynamically and distributed across neurons.

SIGNIFICANCE STATEMENT

Natural scenes often contain multiple entities. The ability to segment visual scenes into distinct objects and surfaces is fundamental to sensory processing and is crucial for generating the perception of our environment. Because cortical neurons are broadly tuned to a given visual feature, segmenting two stimuli that differ only slightly is a challenge for the visual system. In this study, we discovered that many neurons in the visual cortex are capable of representing individual components of slightly different stimuli by selectively and nonlinearly pooling the responses elicited by the stimulus components. We also show for the first time that the neural representation of individual stimulus components developed over a period of ∼70-100 ms, revealing a dynamic process of image segmentation.

Spatial precision of population activity in primate area MT

The middle temporal (MT) area is a cortical area integral to the "where" pathway of primate visual processing, signaling the movement and position of objects in the visual world. The receptive field of a single MT neuron is sensitive to the direction of object motion but is too large to signal precise spatial position. Here, we asked if the activity of MT neurons could be combined to support the high spatial precision required in the where pathway. With the use of multielectrode arrays, we recorded simultaneously neural activity at 24-65 sites in area MT of anesthetized marmoset monkeys. We found that although individual receptive fields span more than 5° of the visual field, the combined population response can support fine spatial discriminations (<0.2°). This is because receptive fields at neighboring sites overlapped substantially, and changes in spatial position are therefore projected onto neural activity in a large ensemble of neurons. This fine spatial discrimination is supported primarily by neurons with receptive fields flanking the target locations. Population performance is degraded (by 13-22%) when correlations in neural activity are ignored, further reflecting the contribution of population neural interactions. Our results show that population signals can provide high spatial precision despite large receptive fields, allowing area MT to represent both the motion and the position of objects in the visual world.

Enhanced integration of motion information in children with autism

To judge the overall direction of a shoal of fish or a crowd of people, observers must integrate motion signals across space and time. The limits on our ability to pool motion have largely been established using the motion coherence paradigm, in which observers report the direction of coherently moving dots amid randomly moving noise dots. Poor performance by autistic individuals on this task has widely been interpreted as evidence of disrupted integrative processes. Critically, however, motion coherence thresholds are not necessarily limited only by pooling. They could also be limited by imprecision in estimating the direction of individual elements or by difficulties segregating signal from noise. Here, 33 children with autism 6-13 years of age and 33 age- and ability-matched typical children performed a more robust task reporting mean dot direction both in the presence and the absence of directional variability alongside a standard motion coherence task. Children with autism were just as sensitive to directional differences as typical children when all elements moved in the same direction (no variability). However, remarkably, children with autism were more sensitive to the average direction in the presence of directional variability, providing the first evidence of enhanced motion integration in autism. Despite this improved averaging ability, children with autism performed comparably to typical children in the motion coherence task, suggesting that their motion coherence thresholds may be limited by reduced segregation of signal from noise. Although potentially advantageous under some conditions, increased integration may lead to feelings of "sensory overload" in children with autism.

Functional MRI of visual responses in the awake, behaving marmoset

The visual brain is composed of interconnected subcortical and cortical structures that receive and process image information originating in the retina. The visual system of nonhuman primates, in particular macaques, has been studied in great detail in order to elucidate principles of human sensation and perception. The common marmoset (Callithrix jacchus) is a small New World monkey of growing interest as a primate model for neuroscience. Marmosets have advantages over macaques because of their small size, lissencephalic cortex, and growing potential for viral and genetic manipulations. Previous anatomical studies and electrophysiological recordings in anesthetized marmosets have shown that this species' cortical visual hierarchy closely resembles that of other primates, including humans. Until now, however, there have been no attempts to systematically study visual responses throughout the marmoset brain using fMRI. Here we show that awake marmosets readily learn to carry out a simple visual task inside the bore of an MRI scanner during functional mapping experiments. Functional scanning at 500 μm in-plane resolution in a 30 cm horizontal bore at 7 T revealed robust positive blood oxygenation level-dependent (BOLD) fMRI responses to visual stimuli throughout visual cortex and associated subcortical areas. Nonvisual sensory areas showed negative contrasts to visual stimuli compared to the fixation dot only baseline. Structured images of objects and faces led to stronger responses than scrambled control images at stages beyond early visual cortex. Our study establishes fMRI mapping of visual responses in awake, behaving marmosets as a straightforward and valuable tool for assessing the functional organization of the primate brain at high resolution.

Emergence of complex wave patterns in primate cerebral cortex

Slow brain rhythms are attributed to near-simultaneous (synchronous) changes in activity in neuron populations in the brain. Because they are slow and widespread, synchronous rhythms have not been considered crucial for information processing in the waking state. Here we adapted methods from turbulence physics to analyze δ-band (1-4 Hz) rhythms in local field potential (LFP) activity, in multielectrode recordings from cerebral cortex in anesthetized marmoset monkeys. We found that synchrony contributes only a small fraction (less than one-fourth) to the local spatiotemporal structure of δ-band signals. Rather, δ-band activity is dominated by propagating plane waves and spatiotemporal structures, which we call complex waves. Complex waves are manifest at submillimeter spatial scales, and millisecond-range temporal scales. We show that complex waves can be characterized by their relation to phase singularities within local nerve cell networks. We validate the biological relevance of complex waves by showing that nerve cell spike rates are higher in presence of complex waves than in the presence of synchrony and that there are nonrandom patterns of evolution from one type of complex wave to another. We conclude that slow brain rhythms predominantly indicate spatiotemporally organized activity in local nerve cell circuits, not synchronous activity within and across brain regions.

The marmoset monkey as a model for visual neuroscience

The common marmoset (Callithrix jacchus) has been valuable as a primate model in biomedical research. Interest in this species has grown recently, in part due to the successful demonstration of transgenic marmosets. Here we examine the prospects of the marmoset model for visual neuroscience research, adopting a comparative framework to place the marmoset within a broader evolutionary context. The marmoset's small brain bears most of the organizational features of other primates, and its smooth surface offers practical advantages over the macaque for areal mapping, laminar electrode penetration, and two-photon and optical imaging. Behaviorally, marmosets are more limited at performing regimented psychophysical tasks, but do readily accept the head restraint that is necessary for accurate eye tracking and neurophysiology, and can perform simple discriminations. Their natural gaze behavior closely resembles that of other primates, with a tendency to focus on objects of social interest including faces. Their immaturity at birth and routine twinning also makes them ideal for the study of postnatal visual development. These experimental factors, together with the theoretical advantages inherent in comparing anatomy, physiology, and behavior across related species, make the marmoset an excellent model for visual neuroscience.

Functional mapping of face-selective regions in the extrastriate visual cortex of the marmoset

The cerebral cortex of humans and macaques has specialized regions for processing faces and other visual stimulus categories. It is unknown whether a similar functional organization exists in New World monkeys, such as the common marmoset (Callithrix jacchus), a species of growing interest as a primate model in neuroscience. To address this question, we measured selective neural responses in the brain of four awake marmosets trained to fix their gaze upon images of faces, bodies, objects, and control patterns. In two of the subjects, we measured high gamma-range field potentials from electrocorticography arrays implanted over a large portion of the occipital and inferotemporal cortex. In the other two subjects, we measured BOLD fMRI responses across the entire brain. Both techniques revealed robust, regionally specific patterns of category-selective neural responses. We report that at least six face-selective patches mark the occipitotemporal pathway of the marmoset, with the most anterior patches showing the strongest preference for faces over other stimuli. The similar appearance of these patches to previous findings in macaques and humans, including their apparent arrangement in two parallel pathways, suggests that core elements of the face processing network were present in the common anthropoid primate ancestor living ∼35 million years ago. The findings also identify the marmoset as a viable animal model system for studying specialized neural mechanisms related to high-level social visual perception in humans.

A simpler primate brain: the visual system of the marmoset monkey

Humans are diurnal primates with high visual acuity at the center of gaze. Although primates share many similarities in the organization of their visual centers with other mammals, and even other species of vertebrates, their visual pathways also show unique features, particularly with respect to the organization of the cerebral cortex. Therefore, in order to understand some aspects of human visual function, we need to study non-human primate brains. Which species is the most appropriate model? Macaque monkeys, the most widely used non-human primates, are not an optimal choice in many practical respects. For example, much of the macaque cerebral cortex is buried within sulci, and is therefore inaccessible to many imaging techniques, and the postnatal development and lifespan of macaques are prohibitively long for many studies of brain maturation, plasticity, and aging. In these and several other respects the marmoset, a small New World monkey, represents a more appropriate choice. Here we review the visual pathways of the marmoset, highlighting recent work that brings these advantages into focus, and identify where additional work needs to be done to link marmoset brain organization to that of macaques and humans. We will argue that the marmoset monkey provides a good subject for studies of a complex visual system, which will likely allow an important bridge linking experiments in animal models to humans.

Motion-defined surface segregation in human visual cortex

Surface segregation provides an efficient way to parse the visual scene for perceptual analysis. Here, we investigated the segregation of a bivectorial motion display into transparent surfaces through a psychophysical task and fMRI. We found that perceptual transparency correlated with neural activity in the early areas of the visual cortex, suggesting these areas may be involved in the segregation of motion-defined surfaces. Two oppositely rotating, uniquely colored random dot kinematograms (RDKs) were presented either sequentially or in a spatially interleaved manner, displayed at varying alternation frequencies. Participants reported the color and rotation direction pairing of the RDKs in the psychophysical task. The spatially interleaved display generated the percept of motion transparency across the range of frequencies tested, yielding ceiling task performance. At high alternation frequencies, performance on the sequential display also approached ceiling, indicative of perceived transparency. However, transparency broke down in lower alternation frequency sequential displays, producing performance close to chance. A corresponding pattern mirroring the psychophysical data was also evident in univariate and multivariate analyses of the fMRI BOLD activity in visual cortical areas V1, V2, V3, V3AB, hV4, and V5/MT+. Using gray RDKs, we found significant presentation by frequency interactions in most areas; differences in BOLD signal between presentation types were significant only at the lower alternation frequency. Multivariate pattern classification was similarly unable to discriminate between presentation types at the higher frequency. This study provides evidence that early visual cortex may code for motion-defined surface segregation, which in turn may enable perceptual transparency.