Cortical circuits for integration of self-motion and visual-motion signals

Active vision in freely moving marmosets using head-mounted eye tracking

Our understanding of how vision functions as primates actively navigate the real-world is remarkably sparse. As most data have been limited to chaired and typically head-restrained animals, the synergistic interactions of different motor actions/plans inherent to active sensing - e.g. eyes, head, posture, movement, etc. - on visual perception are largely unknown. To address this considerable gap in knowledge, we developed an innovative wireless head-mounted eye tracking system called CEREBRO for small mammals, such as marmoset monkeys. Our system performs Chair-free Eye-Recording using Backpack mounted micROcontrollers. Because eye illumination and environment lighting change continuously in natural contexts, we developed a segmentation artificial neural network to perform robust pupil tracking in these conditions. Leveraging this innovative system to investigate active vision, we demonstrate that although freely-moving marmosets exhibit frequent compensatory eye movements equivalent to other primates, including humans, the predictability of the visual behavior (gaze) is higher when animals are freely-moving relative to when they are head-fixed. Moreover, despite increases in eye/head-motion during locomotion, gaze stabilization remains steady because of an increase in VOR gain during locomotion. These results demonstrate the efficient, dynamic visuo-motor mechanisms and related behaviors that enable stable, high-resolution foveal vision in primates as they explore the natural world.

Layer 1 neocortex: Gating and integrating multidimensional signals

Layer 1 (L1) of the neocortex acts as a nexus for the collection and processing of widespread information. By integrating ascending inputs with extensive top-down activity, this layer likely provides critical information regulating how the perception of sensory inputs is reconciled with expectation. This is accomplished by sorting, directing, and integrating the complex network of excitatory inputs that converge onto L1. These signals are combined with neuromodulatory afferents and gated by the wealth of inhibitory interneurons that either are embedded within L1 or send axons from other cortical layers. Together, these interactions dynamically calibrate information flow throughout the neocortex. This review will primarily focus on L1 within the primary sensory cortex and will use these insights to understand L1 in other cortical areas.

Can mirror self-recognition in mice unpack the neural underpinnings of self-awareness?

In this issue of Neuron, Yokose et al. show that mice groom a mark on their forehead when exposed to a mirror. Comparing this behavior with hominids' helps carve self-awareness into its component parts and explore the neural mechanisms of its shared components.

Cortical Integration of Vestibular and Visual Cues for Navigation, Visual Processing, and Perception

Despite increasing evidence of its involvement in several key functions of the cerebral cortex, the vestibular sense rarely enters our consciousness. Indeed, the extent to which these internal signals are incorporated within cortical sensory representation and how they might be relied upon for sensory-driven decision-making, during, for example, spatial navigation, is yet to be understood. Recent novel experimental approaches in rodents have probed both the physiological and behavioral significance of vestibular signals and indicate that their widespread integration with vision improves both the cortical representation and perceptual accuracy of self-motion and orientation. Here, we summarize these recent findings with a focus on cortical circuits involved in visual perception and spatial navigation and highlight the major remaining knowledge gaps. We suggest that vestibulo-visual integration reflects a process of constant updating regarding the status of self-motion, and access to such information by the cortex is used for sensory perception and predictions that may be implemented for rapid, navigation-related decision-making.

Walking humans and running mice: perception and neural encoding of optic flow during self-motion

Locomotion produces full-field optic flow that often dominates the visual motion inputs to an observer. The perception of optic flow is in turn important for animals to guide their heading and interact with moving objects. Understanding how locomotion influences optic flow processing and perception is therefore essential to understand how animals successfully interact with their environment. Here, we review research investigating how perception and neural encoding of optic flow are altered during self-motion, focusing on locomotion. Self-motion has been found to influence estimation and sensitivity for optic flow speed and direction. Nonvisual self-motion signals also increase compensation for self-driven optic flow when parsing the visual motion of moving objects. The integration of visual and nonvisual self-motion signals largely follows principles of Bayesian inference and can improve the precision and accuracy of self-motion perception. The calibration of visual and nonvisual self-motion signals is dynamic, reflecting the changing visuomotor contingencies across different environmental contexts. Throughout this review, we consider experimental research using humans, non-human primates and mice. We highlight experimental challenges and opportunities afforded by each of these species and draw parallels between experimental findings. These findings reveal a profound influence of locomotion on optic flow processing and perception across species. This article is part of a discussion meeting issue 'New approaches to 3D vision'.

Neuroscience and architecture: Modulating behavior through sensorimotor responses to the built environment

As we move through the world, natural and built environments implicitly guide behavior by appealing to certain sensory and motor dynamics. This process can be motivated by automatic attention to environmental features that resonate with specific sensorimotor responses. This review aims at providing a psychobiological framework describing how environmental features can lead to automated sensorimotor responses through defined neurophysiological mechanisms underlying attention. Through the use of automated processes in subsets of cortical structures, the goal of this framework is to describe on a neuronal level the functional link between the designed environment and sensorimotor responses. By distinguishing between environmental features and sensorimotor responses we elaborate on how automatic behavior employs the environment for sensorimotor adaptation. This is realized through a thalamo-cortical network integrating environmental features with motor aspects of behavior. We highlight the underlying transthalamic transmission from an Enactive and predictive perspective and review recent studies that effectively modulated behavior by systematically manipulating environmental features. We end by suggesting a promising combination of neuroimaging and computational analysis for future studies.

The Development of Receptive Field Tuning Properties in Mouse Binocular Primary Visual Cortex

The mouse primary visual cortex is a model system for understanding the relationship between cortical structure, function, and behavior (Seabrook et al., 2017; Chaplin and Margrie, 2020; Hooks and Chen, 2020; Saleem, 2020; Flossmann and Rochefort, 2021). Binocular neurons in V1 are the cellular basis of binocular vision, which is required for predation (Scholl et al., 2013; Hoy et al., 2016; La Chioma et al., 2020; Berson, 2021; Johnson et al., 2021). The normal development of binocular responses, however, has not been systematically measured. Here, we measure tuning properties of neurons to either eye in awake mice of either sex from eye opening to the closure of the critical period. At eye opening, we find an adult-like fraction of neurons responding to the contralateral-eye stimulation, which are selective for orientation and spatial frequency; few neurons respond to ipsilateral eye, and their tuning is immature. Fraction of ipsilateral-eye responses increases rapidly in the first few days after eye opening and more slowly thereafter, reaching adult levels by critical period closure. Tuning of these responses improves with a similar time course. The development and tuning of binocular responses parallel that of ipsilateral-eye responses. Four days after eye opening, monocular neurons respond to a full range of orientations but become more biased to cardinal orientations. Binocular responses, by contrast, lose their cardinal bias with age. Together, these data provide an in-depth accounting of the development of monocular and binocular responses in the binocular region of mouse V1 using a consistent set of visual stimuli and measurements.SIGNIFICANCE STATEMENT In this manuscript, we present a full accounting of the emergence and refinement of monocular and binocular receptive field tuning properties of thousands of pyramidal neurons in mouse primary visual cortex. Our data reveal new features of monocular and binocular development that revise current models on the emergence of cortical binocularity. Given the recent interest in visually guided behaviors in mice that require binocular vision (e.g., predation), our measures will provide the basis for studies on the emergence of the neural circuitry guiding these behaviors.

Perceptual Awareness of Optic Flows Paced Optimally and Non-optimally to Walking Speed

Previous research suggests that visual processing depends strongly on locomotor activity and is tuned to optic flows consistent with self-motion speed. Here, we used a binocular rivalry paradigm to investigate whether perceptual access to optic flows depends on their optimality in relation to walking velocity. Participants walked at two different speeds on a treadmill while viewing discrepant visualizations of a virtual tunnel in each eye. We hypothesized that visualizations paced appropriately to the walking speeds will be perceived longer than non optimal (too fast/slow) ones. The presented optic flow speeds were predetermined individually in a task based on matching visual speed to both walking velocities. In addition, perceptual preference for optimal optic flows was expected to increase with proprioceptive ability to detect threshold-level changes in walking speed. Whereas faster (more familiar) optic flows showed enhanced access to awareness during faster compared with slower walking conditions, for slower visual flows, only a nonsignificant tendency for the analogous effect was observed. These effects were not dependent on individual proprioceptive sensitivity. Our findings concur with the emerging view that the velocity of one's locomotion is used to calibrate visual perception of self-motion and extend the scope of reported action effects on visual awareness.

Evaluating the sensitivity of functional connectivity measures to motion artifact in resting-state fMRI data

Functional connectivity (FC) networks are typically inferred from resting-state fMRI data using the Pearson correlation between BOLD time series from pairs of brain regions. However, alternative methods of estimating functional connectivity have not been systematically tested for their sensitivity or robustness to head motion artifact. Here, we evaluate the sensitivity of eight different functional connectivity measures to motion artifact using resting-state data from the Human Connectome Project. We report that FC estimated using full correlation has a relatively high residual distance-dependent relationship with motion compared to partial correlation, coherence, and information theory-based measures, even after implementing rigorous methods for motion artifact mitigation. This disadvantage of full correlation, however, may be offset by higher test-retest reliability, fingerprinting accuracy, and system identifiability. FC estimated by partial correlation offers the best of both worlds, with low sensitivity to motion artifact and intermediate system identifiability, with the caveat of low test-retest reliability and fingerprinting accuracy. We highlight spatial differences in the sub-networks affected by motion with different FC metrics. Further, we report that intra-network edges in the default mode and retrosplenial temporal sub-networks are highly correlated with motion in all FC methods. Our findings indicate that the method of estimating functional connectivity is an important consideration in resting-state fMRI studies and must be chosen carefully based on the parameters of the study.

Forward optic flow is prioritised in visual awareness independently of walking direction

When two different images are presented separately to each eye, one experiences smooth transitions between them-a phenomenon called binocular rivalry. Previous studies have shown that exposure to signals from other senses can enhance the access of stimulation-congruent images to conscious perception. However, despite our ability to infer perceptual consequences from bodily movements, evidence that action can have an analogous influence on visual awareness is scarce and mainly limited to hand movements. Here, we investigated whether one's direction of locomotion affects perceptual access to optic flow patterns during binocular rivalry. Participants walked forwards and backwards on a treadmill while viewing highly-realistic visualisations of self-motion in a virtual environment. We hypothesised that visualisations congruent with walking direction would predominate in visual awareness over incongruent ones, and that this effect would increase with the precision of one's active proprioception. These predictions were not confirmed: optic flow consistent with forward locomotion was prioritised in visual awareness independently of walking direction and proprioceptive abilities. Our findings suggest the limited role of kinaesthetic-proprioceptive information in disambiguating visually perceived direction of self-motion and indicate that vision might be tuned to the (expanding) optic flow patterns prevalent in everyday life.

Analysis of segmentation ontology reveals the similarities and differences in connectivity onto L2/3 neurons in mouse V1

Quantitatively comparing brain-wide connectivity of different types of neuron is of vital importance in understanding the function of the mammalian cortex. Here we have designed an analytical approach to examine and compare datasets from hierarchical segmentation ontologies, and applied it to long-range presynaptic connectivity onto excitatory and inhibitory neurons, mainly located in layer 2/3 (L2/3), of mouse primary visual cortex (V1). We find that the origins of long-range connections onto these two general cell classes-as well as their proportions-are quite similar, in contrast to the inputs on to a cell type in L6. These anatomical data suggest that distal inputs received by the general excitatory and inhibitory classes of neuron in L2/3 overlap considerably.

The Untapped Potential of Virtual Reality in Rehabilitation of Balance and Gait in Neurological Disorders

Dynamic systems theory transformed our understanding of motor control by recognizing the continual interaction between the organism and the environment. Movement could no longer be visualized simply as a response to a pattern of stimuli or as a demonstration of prior intent; movement is context dependent and is continuously reshaped by the ongoing dynamics of the world around us. Virtual reality is one methodological variable that allows us to control and manipulate that environmental context. A large body of literature exists to support the impact of visual flow, visual conditions, and visual perception on the planning and execution of movement. In rehabilitative practice, however, this technology has been employed mostly as a tool for motivation and enjoyment of physical exercise. The opportunity to modulate motor behavior through the parameters of the virtual world is often ignored in practice. In this article we present the results of experiments from our laboratories and from others demonstrating that presenting particular characteristics of the virtual world through different sensory modalities will modify balance and locomotor behavior. We will discuss how movement in the virtual world opens a window into the motor planning processes and informs us about the relative weighting of visual and somatosensory signals. Finally, we discuss how these findings should influence future treatment design.

Spatial navigation signals in rodent visual cortex

During navigation, animals integrate sensory information with body movements to guide actions. The impact of both navigational and movement-related signals on cortical visual information processing remains largely unknown. We review recent studies in awake rodents that have revealed navigation-related signals in the primary visual cortex (V1) including speed, distance travelled and head-orienting movements. Both cortical and subcortical inputs convey self-motion related information to V1 neurons: for example, top-down inputs from secondary motor and retrosplenial cortices convey information about head movements and spatial expectations. Within V1, subtypes of inhibitory neurons are critical for the integration of navigation-related and visual signals. We conclude with potential functional roles of navigation-related signals in V1 including gain control, motor error signals and predictive coding.

Rapid Recalibration of Peri-Personal Space: Psychophysical, Electrophysiological, and Neural Network Modeling Evidence

Interactions between individuals and the environment occur within the peri-personal space (PPS). The encoding of this space plastically adapts to bodily constraints and stimuli features. However, these remapping effects have not been demonstrated on an adaptive time-scale, trial-to-trial. Here, we test this idea first via a visuo-tactile reaction time (RT) paradigm in augmented reality where participants are asked to respond as fast as possible to touch, as visual objects approach them. Results demonstrate that RTs to touch are facilitated as a function of visual proximity, and the sigmoidal function describing this facilitation shifts closer to the body if the immediately precedent trial had indexed a smaller visuo-tactile disparity. Next, we derive the electroencephalographic correlates of PPS and demonstrate that this multisensory measure is equally shaped by recent sensory history. Finally, we demonstrate that a validated neural network model of PPS is able to account for the present results via a simple Hebbian plasticity rule. The present findings suggest that PPS encoding remaps on a very rapid time-scale and, more generally, that it is sensitive to sensory history, a key feature for any process contextualizing subsequent incoming sensory information (e.g., a Bayesian prior).

Two Distinct Types of Eye-Head Coupling in Freely Moving Mice

Animals actively interact with their environment to gather sensory information. There is conflicting evidence about how mice use vision to sample their environment. During head restraint, mice make rapid eye movements coupled between the eyes, similar to conjugate saccadic eye movements in humans. However, when mice are free to move their heads, eye movements are more complex and often non-conjugate, with the eyes moving in opposite directions. We combined head and eye tracking in freely moving mice and found both observations are explained by two eye-head coupling types, associated with vestibular mechanisms. The first type comprised non-conjugate eye movements, which compensate for head tilt changes to maintain a similar visual field relative to the horizontal ground plane. The second type of eye movements was conjugate and coupled to head yaw rotation to produce a "saccade and fixate" gaze pattern. During head-initiated saccades, the eyes moved together in the head direction but during subsequent fixation moved in the opposite direction to the head to compensate for head rotation. This saccade and fixate pattern is similar to humans who use eye movements (with or without head movement) to rapidly shift gaze but in mice relies on combined head and eye movements. Both couplings were maintained during social interactions and visually guided object tracking. Even in head-restrained mice, eye movements were invariably associated with attempted head motion. Our results reveal that mice combine head and eye movements to sample their environment and highlight similarities and differences between eye movements in mice and humans.