Visual and proprioceptive recalibrations after exposure to a visuomotor rotation

Multisensory perception depends on the reliability of the type of judgment

The brain engages the processes of multisensory integration and recalibration to deal with discrepant multisensory signals. These processes consider the reliability of each sensory input, with the more reliable modality receiving the stronger weight. Sensory reliability is typically assessed via the variability of participants' judgments, yet these can be shaped by factors both external and internal to the nervous system. For example, motor noise and participant's dexterity with the specific response method contribute to judgment variability, and different response methods applied to the same stimuli can result in different estimates of sensory reliabilities. Here we ask how such variations in reliability induced by variations in the response method affect multisensory integration and sensory recalibration, as well as motor adaptation, in a visuomotor paradigm. Participants performed center-out hand movements and were asked to judge the position of the hand or rotated visual feedback at the movement end points. We manipulated the variability, and thus the reliability, of repeated judgments by asking participants to respond using either a visual or a proprioceptive matching procedure. We find that the relative weights of visual and proprioceptive signals, and thus the asymmetry of multisensory integration and recalibration, depend on the reliability modulated by the judgment method. Motor adaptation, in contrast, was insensitive to this manipulation. Hence, the outcome of multisensory binding is shaped by the noise introduced by sensorimotor processing, in line with perception and action being intertwined.NEW & NOTEWORTHY Our brain tends to combine multisensory signals based on their respective reliability. This reliability depends on sensory noise in the environment, noise in the nervous system, and, as we show here, variability induced by the specific judgment procedure.

Minimal impact of proprioceptive loss on implicit sensorimotor adaptation and perceived movement outcome

Implicit sensorimotor adaptation keeps our movements well-calibrated amid changes in the body and environment. We have recently postulated that implicit adaptation is driven by a perceptual error: the difference between the desired and perceived movement outcome. According to this perceptual re-alignment model, implicit adaptation ceases when the perceived movement outcome - a multimodal percept determined by a prior belief conveying the intended action, the motor command, and feedback from proprioception and vision - is aligned with the desired movement outcome. Here, we examined the role of proprioception in implicit motor adaptation and perceived movement outcome by examining individuals who lack proprioception. We used a modified visuomotor rotation task designed to isolate implicit adaptation and probe perceived outcome throughout the experiment. Surprisingly, implicit adaptation and perceived outcome were minimally impacted by deafferentation, posing a challenge to the perceptual re-alignment model of implicit adaptation.

Different time scales of common-cause evidence shape multisensory integration, recalibration and motor adaptation

Perceptual coherence in the face of discrepant multisensory signals is achieved via the processes of multisensory integration, recalibration and sometimes motor adaptation. These supposedly operate on different time scales, with integration reducing immediate sensory discrepancies and recalibration and motor adaptation reflecting the cumulative influence of their recent history. Importantly, whether discrepant signals are bound during perception is guided by the brains' inference of whether they originate from a common cause. When combined, these two notions lead to the hypothesis that the time scales on which integration and recalibration (or motor adaptation) operate are associated with different time scales of evidence about a common cause underlying two signals. We tested this prediction in a well-established visuo-motor paradigm, in which human participants performed visually guided hand movements. The kinematic correlation between hand and cursor movements indicates their common origin, which allowed us to manipulate the common-cause evidence by titrating this correlation. Specifically, we dissociated hand and cursor signals during individual movements while preserving their correlation across the series of movement endpoints. Following our hypothesis, this manipulation reduced integration compared with a condition in which visual and proprioceptive signals were perfectly correlated. In contrast, recalibration and motor adaption were not affected by this manipulation. This supports the notion that multisensory integration and recalibration deal with sensory discrepancies on different time scales guided by common-cause evidence: Integration is prompted by local common-cause evidence and reduces immediate discrepancies, whereas recalibration and motor adaptation are prompted by global common-cause evidence and reduce persistent discrepancies.

Changes in body perception following virtual object manipulation are accompanied by changes of the internal reference scale

Changes in body perception often arise when observers are confronted with related yet discrepant multisensory signals. Some of these effects are interpreted as outcomes of sensory integration of various signals, whereas related biases are ascribed to learning-dependent recalibration of coding individual signals. The present study explored whether the same sensorimotor experience entails changes in body perception that are indicative of multisensory integration and those that indicate recalibration. Participants enclosed visual objects by a pair of visual cursors controlled by finger movements. Then either they judged their perceived finger posture (indicating multisensory integration) or they produced a certain finger posture (indicating recalibration). An experimental variation of the size of the visual object resulted in systematic and opposite biases of the perceived and produced finger distances. This pattern of results is consistent with the assumption that multisensory integration and recalibration had a common origin in the task we used.

Short-term effects of visuomotor discrepancies on multisensory integration, proprioceptive recalibration, and motor adaptation

Information about the position of our hand is provided by multisensory signals that are often not perfectly aligned. Discrepancies between the seen and felt hand position or its movement trajectory engage the processes of 1) multisensory integration, 2) sensory recalibration, and 3) motor adaptation, which adjust perception and behavioral responses to apparently discrepant signals. To foster our understanding of the coemergence of these three processes, we probed their short-term dependence on multisensory discrepancies in a visuomotor task that has served as a model for multisensory perception and motor control previously. We found that the well-established integration of discrepant visual and proprioceptive signals is tied to the immediate discrepancy and independent of the outcome of the integration of discrepant signals in immediately preceding trials. However, the strength of integration was context dependent, being stronger in an experiment featuring stimuli that covered a smaller range of visuomotor discrepancies (±15°) compared with one covering a larger range (±30°). Both sensory recalibration and motor adaptation for nonrepeated movement directions were absent after two bimodal trials with same or opposite visuomotor discrepancies. Hence our results suggest that short-term sensory recalibration and motor adaptation are not an obligatory consequence of the integration of preceding discrepant multisensory signals.NEW & NOTEWORTHY The functional relation between multisensory integration and recalibration remains debated. We here refute the notion that they coemerge in an obligatory manner and support the hypothesis that they serve distinct goals of perception.

Understanding implicit sensorimotor adaptation as a process of proprioceptive re-alignment

Multiple learning processes contribute to successful goal-directed actions in the face of changing physiological states, biomechanical constraints, and environmental contexts. Amongst these processes, implicit sensorimotor adaptation is of primary importance, ensuring that movements remain well-calibrated and accurate. A large body of work on reaching movements has emphasized how adaptation centers on an iterative process designed to minimize visual errors. The role of proprioception has been largely neglected, thought to play a passive role in which proprioception is affected by the visual error but does not directly contribute to adaptation. Here, we present an alternative to this visuo-centric framework, outlining a model in which implicit adaptation acts to minimize a proprioceptive error, the distance between the perceived hand position and its intended goal. This proprioceptive re-alignment model (PReMo) is consistent with many phenomena that have previously been interpreted in terms of learning from visual errors, and offers a parsimonious account of numerous unexplained phenomena. Cognizant that the evidence for PReMo rests on correlational studies, we highlight core predictions to be tested in future experiments, as well as note potential challenges for a proprioceptive-based perspective on implicit adaptation.

Visuo-proprioceptive integration and recalibration with multiple visual stimuli

To organize the plethora of sensory signals from our environment into a coherent percept, our brain relies on the processes of multisensory integration and sensory recalibration. We here asked how visuo-proprioceptive integration and recalibration are shaped by the presence of more than one visual stimulus, hence paving the way to study multisensory perception under more naturalistic settings with multiple signals per sensory modality. We used a cursor-control task in which proprioceptive information on the endpoint of a reaching movement was complemented by two visual stimuli providing additional information on the movement endpoint. The visual stimuli were briefly shown, one synchronously with the hand reaching the movement endpoint, the other delayed. In Experiment 1, the judgments of hand movement endpoint revealed integration and recalibration biases oriented towards the position of the synchronous stimulus and away from the delayed one. In Experiment 2 we contrasted two alternative accounts: that only the temporally more proximal visual stimulus enters integration similar to a winner-takes-all process, or that the influences of both stimuli superpose. The proprioceptive biases revealed that integration—and likely also recalibration—are shaped by the superposed contributions of multiple stimuli rather than by only the most powerful individual one.

Precision control for a flexible body representation

Adaptive body representation requires the continuous integration of multisensory inputs within a flexible 'body model' in the brain. The present review evaluates the idea that this flexibility is augmented by the contextual modulation of sensory processing 'top-down'; which can be described as precision control within predictive coding formulations of Bayesian inference. Specifically, I focus on the proposal that an attenuation of proprioception may facilitate the integration of conflicting visual and proprioceptive bodily cues. Firstly, I review empirical work suggesting that the processing of visual vs proprioceptive body position information can be contextualised 'top-down'; for instance, by adopting specific attentional task sets. Building up on this, I review research showing a similar contextualisation of visual vs proprioceptive information processing in the rubber hand illusion and in visuomotor adaptation. Together, the reviewed literature suggests that proprioception, despite its indisputable importance for body perception and action control, can be attenuated top-down (through precision control) to facilitate the contextual adaptation of the brain's body model to novel visual feedback.

Individual differences in proprioception predict the extent of implicit sensorimotor adaptation

Recent studies have revealed an upper bound in motor adaptation, beyond which other learning systems may be recruited. The factors determining this upper bound are poorly understood. The multisensory integration hypothesis states that this limit arises from opposing responses to visual and proprioceptive feedback. As individuals adapt to a visual perturbation, they experience an increasing proprioceptive error in the opposite direction, and the upper bound is the point where these two error signals reach an equilibrium. Assuming that visual and proprioceptive feedback are weighted according to their variability, there should be a correlation between proprioceptive variability and the limits of adaptation. Alternatively, the proprioceptive realignment hypothesis states that the upper bound arises when the (visually biased) sensed hand position realigns with the expected sensed position (target). When a visuo-proprioceptive discrepancy is introduced, the sensed hand position is biased toward the visual cursor, and the adaptive system counteracts this discrepancy by driving the hand away from the target. This hypothesis predicts a correlation between the size of the proprioceptive shift and the upper bound of adaptation. We tested these two hypotheses by considering natural variation in proprioception and motor adaptation across individuals. We observed a modest, yet reliable correlation between the upper bound of adaptation with both proprioceptive measures (variability and shift). Although the results do not clearly favor one hypothesis over the other, they underscore the critical role of proprioception in sensorimotor adaptation.NEW & NOTEWORTHY Although the sensorimotor system uses sensory feedback to remain calibrated, this learning process is constrained, limited by the maximum degree of plasticity. The factors determining this limit remain elusive. Guided by two hypotheses, we show that individual differences in the upper bound of adaptation in response to a visual perturbation can be predicted by the bias and variability in proprioception. These results underscore the critical, but often neglected role of proprioception in human motor learning.

Suppression of mutually incompatible proprioceptive and visual action effects in tool use

Movements of a tool typically diverge from the movements of the hand manipulating that tool, such as when operating a pivotal lever where tool and hand move in opposite directions. Previous studies suggest that humans are often unaware of the position or movements of their effective body part (mostly the hand) in such situations. It has been suggested that this might be due to a "haptic neglect" of bodily sensations to decrease the interference of representations of body and tool movements. However, in principle this interference could also be decreased by neglecting sensations regarding the tool and focusing instead on body movements. While in most tool use situations the tool-related action effects are task-relevant and thus suppression of body-related rather than tool-related sensations is more beneficial for successful goal achievement, we manipulated this task-relevance in a controlled experiment. The results showed that visual, tool-related effect representations can be suppressed just as proprioceptive, body-related ones in situations where effect representations interfere, given that task-relevance of body-related effects is increased relative to tool-related ones.

A condition that produces sensory recalibration and abolishes multisensory integration

We examined the influence of extended exposure to a visuomotor rotation, which induces both motor adaptation and sensory recalibration, on (partial) multisensory integration in a cursor-control task. Participants adapted to a 30° (adaptation condition) or 0° (control condition) visuomotor rotation by making center-out movements to remembered targets. In subsequent test trials of sensory integration, they made center-out movements with variable visuomotor rotations and judged the position of hand or cursor at the end of these movements. Test trials were randomly embedded among twice the number of maintenance trials with 30° or 0° rotation. The biases of perceived hand (or cursor) position toward the cursor (or hand) position were measured. We found motor adaptation together with proprioceptive and visual recalibrations in the adaptation condition. Unexpectedly, multisensory integration was absent in both the adaptation and control condition. The absence stemmed from the extensive experience of constant visuomotor rotations of 30° or 0°, which probably produced highly precise predictions of the visual consequences of hand movements. The frequently confirmed predictions then dominated the estimate of the visual movement consequences, leaving no influence of the actual visuomotor rotations in the minority of test trials. Conversely, multisensory integration was present for sensed hand positions when these were indirectly assessed from movement characteristics, indicating that the relative weighting of discrepant estimates of hand position was different for motor control. The existence of a condition that abolishes multisensory integration while keeping sensory recalibration suggests that mechanisms that reduce sensory discrepancies (partly) differ between integration and recalibration.

Active inference under visuo-proprioceptive conflict: Simulation and empirical results

It has been suggested that the brain controls hand movements via internal models that rely on visual and proprioceptive cues about the state of the hand. In active inference formulations of such models, the relative influence of each modality on action and perception is determined by how precise (reliable) it is expected to be. The ‘top-down’ affordance of expected precision to a particular sensory modality is associated with attention. Here, we asked whether increasing attention to (i.e., the precision of) vision or proprioception would enhance performance in a hand-target phase matching task, in which visual and proprioceptive cues about hand posture were incongruent. We show that in a simple simulated agent—based on predictive coding formulations of active inference—increasing the expected precision of vision or proprioception improved task performance (target matching with the seen or felt hand, respectively) under visuo-proprioceptive conflict. Moreover, we show that this formulation captured the behaviour and self-reported attentional allocation of human participants performing the same task in a virtual reality environment. Together, our results show that selective attention can balance the impact of (conflicting) visual and proprioceptive cues on action—rendering attention a key mechanism for a flexible body representation for action.