Cortical neurons combine visual cues about self-movement

Causal inference accounts for heading perception in the presence of object motion

The brain infers our spatial orientation and properties of the world from ambiguous and noisy sensory cues. Judging self-motion (heading) in the presence of independently moving objects poses a challenging inference problem because the image motion of an object could be attributed to movement of the object, self-motion, or some combination of the two. We test whether perception of heading and object motion follows predictions of a normative causal inference framework. In a dual-report task, subjects indicated whether an object appeared stationary or moving in the virtual world, while simultaneously judging their heading. Consistent with causal inference predictions, the proportion of object stationarity reports, as well as the accuracy and precision of heading judgments, depended on the speed of object motion. Critically, biases in perceived heading declined when the object was perceived to be moving in the world. Our findings suggest that the brain interprets object motion and self-motion using a causal inference framework.

Processing of object motion and self-motion in the lateral subdivision of the medial superior temporal area in macaques

Multiple areas of macaque cortex are involved in visual motion processing, but their relative functional roles remain unclear. The medial superior temporal (MST) area is typically divided into lateral (MSTl) and dorsal (MSTd) subdivisions that are thought to be involved in processing object motion and self-motion, respectively. Whereas MSTd has been studied extensively with regard to processing visual and nonvisual self-motion cues, little is known about self-motion signals in MSTl, especially nonvisual signals. Moreover, little is known about how self-motion and object motion signals interact in MSTl and how this differs from interactions in MSTd. We compared the visual and vestibular heading tuning of neurons in MSTl and MSTd using identical stimuli. Our findings reveal that both visual and vestibular heading signals are weaker in MSTl than in MSTd, suggesting that MSTl is less well suited to participate in self-motion perception than MSTd. We also tested neurons in both areas with a variety of combinations of object motion and self-motion. Our findings reveal that vestibular signals improve the separability of coding of heading and object direction in both areas, albeit more strongly in MSTd due to the greater strength of vestibular signals. Based on a marginalization technique, population decoding reveals that heading and object direction can be more effectively dissociated from MSTd responses than MSTl responses. Our findings help to clarify the respective contributions that MSTl and MSTd make to processing of object motion and self-motion, although our conclusions may be somewhat specific to the multipart moving objects that we employed. NEW & NOTEWORTHY Retinal image motion reflects contributions from both the observer's self-motion and the movement of objects in the environment. The neural mechanisms by which the brain dissociates self-motion and object motion remain unclear. This study provides the first systematic examination of how the lateral subdivision of area MST (MSTl) contributes to dissociating object motion and self-motion. We also examine, for the first time, how MSTl neurons represent translational self-motion based on both vestibular and visual cues.

Dissociation of Self-Motion and Object Motion by Linear Population Decoding That Approximates Marginalization

We use visual image motion to judge the movement of objects, as well as our own movements through the environment. Generally, image motion components caused by object motion and self-motion are confounded in the retinal image. Thus, to estimate heading, the brain would ideally marginalize out the effects of object motion (or vice versa), but little is known about how this is accomplished neurally. Behavioral studies suggest that vestibular signals play a role in dissociating object motion and self-motion, and recent computational work suggests that a linear decoder can approximate marginalization by taking advantage of diverse multisensory representations. By measuring responses of MSTd neurons in two male rhesus monkeys and by applying a recently-developed method to approximate marginalization by linear population decoding, we tested the hypothesis that vestibular signals help to dissociate self-motion and object motion. We show that vestibular signals stabilize tuning for heading in neurons with congruent visual and vestibular heading preferences, whereas they stabilize tuning for object motion in neurons with discrepant preferences. Thus, vestibular signals enhance the separability of joint tuning for object motion and self-motion. We further show that a linear decoder, designed to approximate marginalization, allows the population to represent either self-motion or object motion with good accuracy. Decoder weights are broadly consistent with a readout strategy, suggested by recent computational work, in which responses are decoded according to the vestibular preferences of multisensory neurons. These results demonstrate, at both single neuron and population levels, that vestibular signals help to dissociate self-motion and object motion.SIGNIFICANCE STATEMENT The brain often needs to estimate one property of a changing environment while ignoring others. This can be difficult because multiple properties of the environment may be confounded in sensory signals. The brain can solve this problem by marginalizing over irrelevant properties to estimate the property-of-interest. We explore this problem in the context of self-motion and object motion, which are inherently confounded in the retinal image. We examine how diversity in a population of multisensory neurons may be exploited to decode self-motion and object motion from the population activity of neurons in macaque area MSTd.

Navigational path integration by cortical neurons: origins in higher-order direction selectivity

Navigation relies on the neural processing of sensory cues about observer self-movement and spatial location. Neurons in macaque dorsal medial superior temporal cortex (MSTd) respond to visual and vestibular self-movement cues, potentially contributing to navigation and orientation. We moved monkeys on circular paths around a room while recording the activity of MSTd neurons. MSTd neurons show a variety of sensitivities to the monkey's heading direction, circular path through the room, and place in the room. Changing visual cues alters the relative prevalence of those response properties. Disrupting the continuity of self-movement paths through the environment disrupts path selectivity in a manner linked to the time course of single neuron responses. We hypothesize that sensory cues interact with the spatial and temporal integrative properties of MSTd neurons to derive path selectivity for navigational path integration supporting spatial orientation.

Linking sensory neurons to visually guided behavior: relating MST activity to steering in a virtual environment

Many complex behaviors rely on guidance from sensations. To perform these behaviors, the motor system must decode information relevant to the task from the sensory system. However, identifying the neurons responsible for encoding the appropriate sensory information remains a difficult problem for neurophysiologists. A key step toward identifying candidate systems is finding neurons or groups of neurons capable of representing the stimuli adequately to support behavior. A traditional approach involves quantitatively measuring the performance of single neurons and comparing this to the performance of the animal. One of the strongest pieces of evidence in support of a neuronal population being involved in a behavioral task comes from the signals being sufficient to support behavior. Numerous experiments using perceptual decision tasks show that visual cortical neurons in many areas have this property. However, most visually guided behaviors are not categorical but continuous and dynamic. In this article, we review the concept of sufficiency and the tools used to measure neural and behavioral performance. We show how concepts from information theory can be used to measure the ongoing performance of both neurons and animal behavior. Finally, we apply these tools to dorsal medial superior temporal (MSTd) neurons and demonstrate that these neurons can represent stimuli important to navigation to a distant goal. We find that MSTd neurons represent ongoing steering error in a virtual-reality steering task. Although most individual neurons were insufficient to support the behavior, some very nearly matched the animal's estimation performance. These results are consistent with many results from perceptual experiments and in line with the predictions of Mountcastle's "lower envelope principle."

Task contingencies and perceptual strategies shape behavioral effects on neuronal response profiles

We presented optic flow simulating eight directions of self-movement in the ground plane, while monkeys performed delayed match-to-sample tasks, and we recorded dorsal medial superior temporal (MSTd) neuronal activity. Randomly selected sample headings yield smaller test responses to the neuron's preferred heading when it is near the sample's heading direction and larger test responses to the preferred heading when it is far from the sample's heading. Limiting test stimuli to matching or opposite headings suppresses responses to preferred stimuli in both test conditions, whereas focusing on each neuron's preferred vs. antipreferred stimuli enhances responses to the antipreferred stimulus. Match vs. opposite paradigms create bimodal heading profiles shaped by interactions with late delay-period activity. We conclude that task contingencies, determining the prior probabilities of specific stimuli, interact with the monkeys' perceptual strategy for optic flow analysis. These influences shape attentional and working memory effects on the heading direction selectivities and preferences of MSTd neurons.

Driving strategy alters neuronal responses to self-movement: cortical mechanisms of distracted driving

We presented naturalistic combinations of virtual self-movement stimuli while recording neuronal activity in monkey cerebral cortex. Monkeys used a joystick to drive to a straight ahead heading direction guided by either object motion or optic flow. The selected cue dominates neuronal responses, often mimicking responses evoked when that stimulus is presented alone. In some neurons, driving strategy creates selective response additivities. In others, it creates vulnerabilities to the disruptive effects of independently moving objects. Such cue interactions may be related to the disruptive effects of independently moving objects in Alzheimer's disease patients with navigational deficits.