Spatial action-effect binding

Bound by Experience: Updating the Body Representation When Using Virtual Objects

OBJECTIVE

Four web-based experiments investigated flexibility of disembodiment of a virtual object that is no longer actively controlled. Emphasis was on possibilities to modify the timescale of this process.

BACKGROUND

Interactions with virtual objects are commonplace in settings like teleoperation, rehabilitation, and computer-aided design. These objects are quickly integrated into the operator's body schema (embodiment). Less is known about how long such embodiment lasts. Understanding the dynamics of this process is crucial because different applied settings either profit from fast or slow disembodiment.

METHOD

To induce embodiment, participants moved a 2D virtual hand through operating a computer mouse or touchpad. After initial embodiment, participants either stopped or continued moving for a fixed period of time. Embodiment ratings were collected continuously during each trial.

RESULTS

Results across all experiments indicated that embodiment for the virtual hand gradually increased during active use and gradually decreased after stopping to use it. Disembodiment unfolded nearly twice as fast as embodiment and showed a curved decay pattern. These dynamics remained unaffected by anticipation of active control that would be required in an upcoming task.

CONCLUSION

The results highlight the importance of continuously experiencing active control in virtual interactions if aiming at inducing stable embodiment of a virtual object.

APPLICATION

Our findings suggest that applications of virtual disembodiment such as virtual tools or interventions to affect a person's body representation critically depend on continuous updating of sensorimotor experience. However, if switching between virtual objects, for example, during teleoperation or video gaming, after-effects are unlikely to affect performance.

Reliability of on-line visual feedback influences learning of continuous motor task of healthy young adults

A continuous task was used to determine how the reliability of on-line visual feedback during acquisition impacts motor learning. Participants performed a right hand pointing task of a repeated sequence with a visual cursor that was either reliable, moderately unreliable, or largely unreliable. Delayed retention tests were administered 24 h later, as well as intermanual transfer tests (performed with the left hand). A visuospatial transfer test was performed with the same targets' sequence (same visuospatial configuration) while a motor transfer test was performed with the visual mirror of the targets' sequence (same motor patterns). Results showed that pointing was slower and long-term learning disrupted in the largely unreliable visual cursor condition, compared with the reliable and moderately unreliable conditions. Also, analysis of transfers revealed classically better performance on visuospatial transfer than on motor transfer for the reliable condition. However, here we first show that such difference disappears when the cursor was moderately or largely unreliable. Interestingly, these results indicated a difference in the type of sequence coding, depending on the reliability of the on-line visual feedback. This recourse to mixed coding opens up interesting perspectives, as it is known to promote better learning of motor sequences.

Instant disembodiment of virtual body parts

Evidence from multisensory body illusions suggests that body representations may be malleable, for instance, by embodying external objects. However, adjusting body representations to current task demands also implies that external objects become disembodied from the body representation if they are no longer required. In the current web-based study, we induced the embodiment of a two-dimensional (2D) virtual hand that could be controlled by active movements of a computer mouse or on a touchpad. Following initial embodiment, we probed for disembodiment by comparing two conditions: Participants either continued moving the virtual hand or they stopped moving and kept the hand still. Based on theoretical accounts that conceptualize body representations as a set of multisensory bindings, we expected gradual disembodiment of the virtual hand if the body representations are no longer updated through correlated visuomotor signals. In contrast to our prediction, the virtual hand was instantly disembodied as soon as participants stopped moving it. This result was replicated in two follow-up experiments. The observed instantaneous disembodiment might suggest that humans are sensitive to the rapid changes that characterize action and body in virtual environments, and hence adjust corresponding body representations particularly swiftly.

Subjective Distance Estimates and Sense of Agency in Robotic Wheelchair Control

Sense of agency (SoA) refers to an individual’s awareness of their own actions. SoA studies seek to find objective indicators for the feeling of agency. These indicators, being related to the feeling of control, have practical application in vehicle design. However, they have not been investigated for actions related to the agent’s body movement inherent to steering a vehicle. In our study, participants operated a robotic wheelchair under three conditions: active control by a participant, direct control by the experimenter and remote control by the experimenter. In each trial, a participant drove the wheelchair until a sound signal occurred, after which they stopped the wheelchair and estimated the travelled distance. The subjective estimates were significantly greater when participants operated the wheelchair by themselves. This result contrasts with observations under static settings in previous studies. In an additional study on the electroencephalographic response to a sound presented at a random time after movement onset, the observed latencies in the N1 component implied that participants might have a higher sense of control when they drove the wheelchair. The proposed methodology might become useful to indirectly assess the degree of operator control of a vehicle, primarily in the field of rehabilitation technologies.

Exploring the relationship between perceived Action-Outcome distance and Agency: Evidence from temporal binding

Intentional actions are usually accompanied by a sense of agency (SoA), along with a perceived shortening of action-outcome intervals known as the intentional binding (IB) effect. This is at least partly associated with the perceived strength of action-outcome relationships, which have been described in terms of distance (e.g., a 'distancing effect'). Given that actions in the modern world are increasingly distant from their outcomes, the current study aimed to explore the effect of perceived spatial distance on the strength of IB. Participants voluntarily triggered, or passively observed, a circle on a background that was either flat or appeared three-dimensional, and estimated action-outcome delays. Depth cues modified the circle's perceived distance while the circle itself did not change. When viewed on a forced-perspective background, interval estimates increased with apparent distance, but only when outcomes were caused by intentional actions. This suggests that agency is reduced for outcomes that appear further away.

How Action Shapes Body Ownership Momentarily and Throughout the Lifespan

Objects which a human agent controls by efferent activities (such as real or virtual tools) can be perceived by the agent as belonging to his or her body. This suggests that what an agent counts as “body” is plastic, depending on what she or he controls. Yet there are possible limitations for such momentary plasticity. One of these limitations is that sensations stemming from the body (e.g., proprioception) and sensations stemming from objects outside the body (e.g., vision) are not integrated if they do not sufficiently “match”. What “matches” and what does not is conceivably determined by long–term experience with the perceptual changes that body movements typically produce. Children have accumulated less sensorimotor experience than adults have. Consequently, they express higher flexibility to integrate body-internal and body-external signals, independent of their “match” as suggested by rubber hand illusion studies. However, children’s motor performance in tool use is more affected by mismatching body-internal and body-external action effects than that of adults, possibly because of less developed means to overcome such mismatches. We review research on perception-action interactions, multisensory integration, and developmental psychology to build bridges between these research fields. By doing so, we account for the flexibility of the sense of body ownership for actively controlled events and its development through ontogeny. This gives us the opportunity to validate the suggested mechanisms for generating ownership by investigating their effects in still developing and incomplete stages in children. We suggest testable predictions for future studies investigating both body ownership and motor skills throughout the lifespan.

On the relevance of task instructions for the influence of action on perception

The present study explored how task instructions mediate the impact of action on perception. Participants saw a target object while performing finger movements. Then either the size of the target or the size of the adopted finger postures was judged. The target judgment was attracted by the adopted finger posture indicating sensory integration of body-related and visual signals. The magnitude of integration, however, depended on how the task was initially described. It was substantially larger when the experimental instructions indicated that finger movements and the target object relate to the same event than when they suggested that they are unrelated. This outcome highlights the role of causal inference processes in the emergence of action specific influences in perception.

Suppression of mutually incompatible proprioceptive and visual action effects in tool use

Movements of a tool typically diverge from the movements of the hand manipulating that tool, such as when operating a pivotal lever where tool and hand move in opposite directions. Previous studies suggest that humans are often unaware of the position or movements of their effective body part (mostly the hand) in such situations. It has been suggested that this might be due to a "haptic neglect" of bodily sensations to decrease the interference of representations of body and tool movements. However, in principle this interference could also be decreased by neglecting sensations regarding the tool and focusing instead on body movements. While in most tool use situations the tool-related action effects are task-relevant and thus suppression of body-related rather than tool-related sensations is more beneficial for successful goal achievement, we manipulated this task-relevance in a controlled experiment. The results showed that visual, tool-related effect representations can be suppressed just as proprioceptive, body-related ones in situations where effect representations interfere, given that task-relevance of body-related effects is increased relative to tool-related ones.

Spatial action-effect binding depends on type of action-effect transformation

Spatial action-effect binding denotes the mutual attraction between the perceived position of an effector (e.g., one's own hand) and a distal object that is controlled by this effector. Such spatial binding can be construed as an implicit measure of object ownership, thus the belonging of a controlled object to the own body. The current study investigated how different transformations of hand movements (body-internal action component) into movements of a visual object (body-external action component) affect spatial action-effect binding, and thus implicit object ownership. In brief, participants had to bring a cursor on the computer screen into a predefined target position by moving their occluded hand on a tablet and had to estimate their final hand position. In Experiment 1, we found a significantly lower drift of the proprioceptive position of the hand towards the visual object when hand movements were transformed into laterally inverted cursor movements, rather than cursor movements in the same direction. Experiment 2 showed that this reduction reflected an elimination of spatial action-effect binding in the inverted condition. The results are discussed with respect to the prerequisites for an experience of ownership over artificial, noncorporeal objects. Our results show that predictability of an object movement alone is not a sufficient condition for ownership because, depending on the type of transformation, integration of the effector and a distal object can be fully abolished even under conditions of full controllability.

Measuring the Sense of Agency: A French Adaptation and Validation of the Sense of Agency Scale (F-SoAS)

Sense of Agency (SoA) is the subject of growing attention. It corresponds to the capacity to claim authorship over an action, associate specific consequences with a specific action, and it has been claimed to be a key point in the development of consciousness. It can be measured using the Sense of Agency Scale (SoAS), originally proposed by Tapal et al. (2017), who distinguished it into two-factor: Sense of Positive Agency (SoPA) and Sense of Negative Agency (SoNA). This study reports on the first adaptation of the SoAS into another language, French. For this French version of the Sense of Agency Scale (F-SoAS), we analyzed responses from a sample of 517 native French-speakers. Our results indicate that the scale has good psychometric properties. Factor analysis confirms the same two-factor model as Tapal et al. (2017). However, some items were removed due to insufficient loadings with factors, leading to a short version of the scale (7-item). Furthermore, we observed gender differences that are consistent with findings in the literature. Specifically, women report higher SoNA scores and lower SoPA scores than men. We conclude by discussing possible uses and future directions for the scale.

Exploring the time window for causal inference and the multisensory integration of actions and their visual effects

Successful computer use requires the operator to link the movement of the cursor to that of his or her hand. Previous studies suggest that the brain establishes this perceptual link through multisensory integration, whereby the causality evidence that drives the integration is provided by the correlated hand and cursor movement trajectories. Here, we explored the temporal window during which this causality evidence is effective. We used a basic cursor-control task, in which participants performed out-and-back reaching movements with their hand on a digitizer tablet. A corresponding cursor movement could be shown on a monitor, yet slightly rotated by an angle that varied from trial to trial. Upon completion of the backward movement, participants judged the endpoint of the outward hand or cursor movement. The mutually biased judgements that typically result reflect the integration of the proprioceptive information on hand endpoint with the visual information on cursor endpoint. We here manipulated the time period during which the cursor was visible, thereby selectively providing causality evidence either before or after sensory information regarding the to-be-judged movement endpoint was available. Specifically, the cursor was visible either during the outward or backward hand movement (conditions Out and Back, respectively). Our data revealed reduced integration in the condition Back compared with the condition Out, suggesting that causality evidence available before the to-be-judged movement endpoint is more powerful than later evidence in determining how strongly the brain integrates the endpoint information. This finding further suggests that sensory integration is not delayed until a judgement is requested.

A condition that produces sensory recalibration and abolishes multisensory integration

We examined the influence of extended exposure to a visuomotor rotation, which induces both motor adaptation and sensory recalibration, on (partial) multisensory integration in a cursor-control task. Participants adapted to a 30° (adaptation condition) or 0° (control condition) visuomotor rotation by making center-out movements to remembered targets. In subsequent test trials of sensory integration, they made center-out movements with variable visuomotor rotations and judged the position of hand or cursor at the end of these movements. Test trials were randomly embedded among twice the number of maintenance trials with 30° or 0° rotation. The biases of perceived hand (or cursor) position toward the cursor (or hand) position were measured. We found motor adaptation together with proprioceptive and visual recalibrations in the adaptation condition. Unexpectedly, multisensory integration was absent in both the adaptation and control condition. The absence stemmed from the extensive experience of constant visuomotor rotations of 30° or 0°, which probably produced highly precise predictions of the visual consequences of hand movements. The frequently confirmed predictions then dominated the estimate of the visual movement consequences, leaving no influence of the actual visuomotor rotations in the minority of test trials. Conversely, multisensory integration was present for sensed hand positions when these were indirectly assessed from movement characteristics, indicating that the relative weighting of discrepant estimates of hand position was different for motor control. The existence of a condition that abolishes multisensory integration while keeping sensory recalibration suggests that mechanisms that reduce sensory discrepancies (partly) differ between integration and recalibration.

Connecting action control and agency: Does action-effect binding affect temporal binding?

The sense of agency, i.e., the notion that we, as agents, are in control of our own actions and can affect our environment by acting, is an integral part of human volition. Recent work has attempted to ground agency in basic mechanisms of human action control. Along these lines, action-effect binding has been shown to affect explicit judgments of agency. Here, we investigate if such action-effect bindings are also related to temporal binding which is often used as an implicit measure of agency. In two experiments, we found evidence for the establishment of short-term action-effect bindings as well as temporal binding effects. However, the two phenomena were not associated with each other. This finding suggests that the relation of action control and agency is not a simple one, and it adds to the evidence in favor of a dissociation between subjective agency and perceptual biases such as temporal binding.

Impact of action planning on visual and body perception in a virtual grasping task

The present study examined mutual influences of visual and body-related signals during planning of an object-oriented action. Participants were to enclose a visual target object using two cursors controlled by the movements of their fingers. During movement preparation, they were asked to judge either the size of the object or a certain finger distance. Both types of judgments were systematically affected by the transformation of finger movements into the movements of visual cursors. We suggest that these biases are perceptual consequences of sensory integration of visual and body-related signals relating to the same external object.

Control used and control felt: Two sides of the agency coin

Agency has been defined as the sense of ownership and control of our actions, and the metacognition of agency has now been examined in a number of studies. Here we examined the relations between task demands, the feeling of being in control, and the feeling of using control. As task demands increase, we might feel as if we use a lot of control while feeling little control over the task. It therefore seems possible that the amount of control one feels they have used and how much in control one feels are separable components of the metacognition of control. In two experiments, we manipulated task demands and assessed these two aspects of metacognition. The source of task demand differed for the two experiments. In Experiment 1, we manipulated task demands by varying the sizes of targets in an aiming task. As predicted, we found that reports of control used increased, while reports of control felt decreased, for more difficult aiming conditions. In Experiment 2, we found a similar relation using a different source of demand: response conflict. We connect these reports of control to previous investigations of task demand and agency, as well as prominent conceptions of cognitive control.

Effects of Hand and Hemispace on Multisensory Integration of Hand Position and Visual Feedback

The brain generally integrates a multitude of sensory signals to form a unified percept. Even in cursor control tasks, such as reaching while looking at rotated visual feedback on a monitor, visual information on cursor position and proprioceptive information on hand position are partially integrated (sensory coupling), resulting in mutual biases of the perceived positions of cursor and hand. Previous studies showed that the strength of sensory coupling (sum of the mutual biases) depends on the experience of kinematic correlations between hand movements and cursor motions, whereas the asymmetry of sensory coupling (difference between the biases) depends on the relative reliabilities (inverse of variability) of hand-position and cursor-position estimates (reliability rule). Furthermore, the precision of movement control and perception of hand position are known to differ between hands (left, right) and workspaces (ipsilateral, contralateral), and so does the experience of kinematic correlations from daily life activities. Thus, in the present study, we tested whether strength and asymmetry of sensory coupling for the endpoints of reaches in a cursor control task differ between the right and left hand and between ipsilateral and contralateral hemispace. No differences were found in the strength of sensory coupling between hands or between hemispaces. However, asymmetry of sensory coupling was less in ipsilateral than in contralateral hemispace: in ipsilateral hemispace, the bias of the perceived hand position was reduced, which was accompanied by a smaller variability of the estimates. The variability of position estimates of the dominant right hand was also less than for the non-dominant left hand, but this difference was not accompanied by a difference in the asymmetry of sensory coupling – a violation of the reliability rule, probably due a stronger influence of visual information on right-hand movements. According to these results, the long-term effects of the experienced kinematic correlation between hand movements and cursor motions on the strength of sensory coupling are generic and not specific for hemispaces or hands, whereas the effects of relative reliabilities on the asymmetry of sensory coupling are specific for hemispaces but not for hands.

Explicit knowledge of sensory non-redundancy can reduce the strength of multisensory integration

The brain integrates incoming sensory signals to a degree that depends on the signals' redundancy. Redundancy-which is commonly high when signals originate from a common physical object or event-is estimated by the brain from the signals' spatial and/or temporal correspondence. Here we tested whether verbally instructed knowledge of non-redundancy can also be used to reduce the strength of the sensory integration. We used a cursor-control task in which cursor motions in the frontoparallel plane were controlled by hand movements in the horizontal plane, yet with a small and randomly varying visuomotor rotation that created spatial discrepancies between hand and cursor positions. Consistent with previous studies, we found mutual biases in the hand and cursor position judgments, indicating partial sensory integration. The integration was reduced in strength, but not eliminated, after participants were verbally informed about the non-redundancy (i.e., the spatial discrepancies) in the hand and cursor positions. Comparisons with model predictions excluded confounding bottom-up effects of the non-redundancy instruction. Our findings thus show that participants have top-down control over the degree to which they integrate sensory information. Additionally, we found that the magnitude of this top-down modulatory capability is a reliable individual trait. A comparison between participants with and without video-gaming experience tentatively suggested a relation between top-down modulation of integration strength and attentional control.

Reduced multisensory integration of self-initiated stimuli

The processing and perception of stimuli is altered when these stimuli are not passively presented but rather are actively triggered, or "self-initiated", by the participants. For unimodal stimuli, perceptual changes in stimulus timing and intensity have been demonstrated. Initial results have suggested that self-initiation may affect multisensory processing as well. The present study examined the effects of self-initiation on audiovisual integration in the ventriloquism effect (VE), that is, the mislocalization of auditory stimuli toward a spatially displaced visual stimulus. The effects of self-initiation on the VE were investigated with audiovisual stimuli that featured varying degrees of spatial and temporal separation. Stimuli were either triggered by the participants' button press or not, and stimulus onsets were either predictable or not. Arguing from the perspective of Bayesian causal inference models, we hypothesized self-initiation to increase the prior probability of two stimuli being integrated. Contrary to this intuitive assumption, less VE was observed when the stimuli were self-initiated by the participants than when they were externally generated. Since no effects of self-initiation on unimodal processing were observed, these effects must specifically pertain to multisensory processes. Finally, data were fit with a causal inference model, where self-initiation was associated with a reduction of the prior probability to integrate audiovisual stimuli. In conclusion, the presence of a self-initiated motor signal influences audiovisual integration, such that auditory localization is less biased by visual stimuli, which likely depends on top-down signals.

Sensory integration of movements and their visual effects is not enhanced by spatial proximity

Spatial proximity enhances the sensory integration of exafferent position information, likely because it indicates whether the information comes from a single physical source. Does spatial proximity also affect the integration of position information regarding an action (here a hand movement) with that of its visual effect (here a cursor motion), that is, when the sensory information comes from physically distinct objects? In this study, participants made out-and-back hand movements whereby the outward movements were accompanied by corresponding cursor motions on a monitor. Their subsequent judgments of hand or cursor movement endpoints are typically biased toward each other, consistent with an underlying optimal integration mechanism. To study the effect of spatial proximity, we presented the hand and cursor either in orthogonal planes (horizontal and frontal, respectively) or we aligned them in the horizontal plane. We did not find the expected enhanced integration strength in the latter spatial condition. As a secondary question we asked whether spatial transformations required for the position judgments (i.e., horizontal to frontal or vice versa) could be the origin of previously observed suboptimal variances of the integrated hand and cursor position judgments. We found, however, that the suboptimality persisted when spatial transformations were omitted (i.e., with the hand and cursor in the same plane). Our findings thus clearly show that the integration of actions with their visual effects is, at least for cursor control, independent of spatial proximity.

Optimal integration of actions and their visual effects is based on both online and prior causality evidence

The brain needs to identify redundant sensory signals in order to integrate them optimally. The identification process, referred to as causal inference, depends on the spatial and temporal correspondence of the incoming sensory signals ('online sensory causality evidence') as well as on prior expectations regarding their causal relation. We here examine whether the same causal inference process underlies spatial integration of actions and their visual consequences. We used a basic cursor-control task for which online sensory causality evidence is provided by the correlated hand and cursor movements, and prior expectations are formed by everyday experience of such correlated movements. Participants made out-and-back movements and subsequently judged the hand or cursor movement endpoints. In one condition, we omitted the online sensory causality evidence by showing the cursor only at the movement endpoint. The integration strength was lower than in conditions where the cursor was visible during the outward movement, but a substantial level of integration persisted. These findings support the hypothesis that the binding of actions and their visual consequences is based on the general mechanism of optimal integration, and they specifically show that such binding can occur even if it is previous experience only that identifies the action consequence.

Kinematic cross-correlation induces sensory integration across separate objects

In a basic cursor-control task, the perceived positions of the hand and the cursor are biased towards each other. We recently found that this phenomenon conforms to the reliability-based weighting mechanism of optimal multisensory integration. This indicates that optimal integration is not restricted to sensory signals originating from a single source, as is the prevailing view, but that it also applies to separate objects that are connected by a kinematic relation (i.e. hand and cursor). In the current study, we examined which aspects of the kinematic relation are crucial for eliciting the sensory integration: (i) the cross-correlation between kinematic variables of the hand and cursor trajectories, and/or (ii) an internal model of the hand-cursor kinematic transformation. Participants made out-and-back movements from the centre of a semicircular workspace to its boundary, after which they judged the position where either their hand or the cursor hit the boundary. We analysed the position biases and found that the integration was strong in a condition with high kinematic correlations (a straight hand trajectory was mapped to a straight cursor trajectory), that it was significantly reduced for reduced kinematic correlations (a straight hand trajectory was transformed into a curved cursor trajectory) and that it was not affected by the inability to acquire an internal model of the kinematic transformation (i.e. by the trial-to-trial variability of the cursor curvature). These findings support the idea that correlations play a crucial role in multisensory integration irrespective of the number of sensory sources involved.

Dissociating explicit and implicit measures of sensed hand position in tool use: Effect of relative frequency of judging different objects

In a cursor-control task, the sensed positions of cursor and hand are biased toward each other. We previously found different characteristics of implicit and explicit measures of the bias of sensed hand position toward the position of the cursor, suggesting the existence of distinct neural representations. Here we further explored differences between the two types of measure by varying the proportions of trials with explicit hand-position (H) and cursor-position (C) judgments (C20:H80, C50:H50, and C80:H20). In each trial, participants made a reaching movement to a remembered target, with the visual feedback being rotated randomly, and subsequently they judged the hand or the cursor position. Both the explicitly and implicitly measured biases of sensed hand position were stronger with a low proportion (C80:H20) than with a high proportion (C20:H80) of hand-position judgments, suggesting that both measures place more weight on the sensory modality relevant for the more frequent judgment. With balanced proportions of such judgments (C50:H50), the explicitly assessed biases were similar to those observed with a high proportion of cursor-position judgments (C80:H20), whereas the implicitly assessed biases were similar to those observed with a high proportion of hand-position judgments (C20:H80). Because strong weights of cursor-position or hand-position information may be difficult to increase further but are easy to reduce, the findings suggest that the implicit measure of the bias of sensed hand position places a relatively stronger weight on proprioceptive hand-position information, which is increased no further by a high proportion of hand-position judgments. Conversely, the explicit measure places a relatively stronger weight on visual cursor-position information.

Contrasting effects of adaptation to a visuomotor rotation on explicit and implicit measures of sensory coupling

We previously investigated sensory coupling of the sensed positions of cursor and hand in a cursor-control task and found differential characteristics of implicit and explicit measures of the bias of sensed hand position toward the position of the cursor. The present study further tested whether adaptation to a visuomotor rotation differentially affects these two measures. Participants made center-out reaching movements to remembered targets while looking at a rotated feedback cursor. After sets of practice trials with constant (adaptation condition) or random (control condition) visuomotor rotations, test trials served to assess sensory coupling. In these trials, participants judged the position of the hand at the end of the center-out movement, and the deviation of these judgments from the physical hand positions served as explicit measure of the bias of sensed hand position toward the position of the cursor, whereas the implicit measure was based on the direction of the return movement. The results showed that inter-individual variability of explicitly assessed biases of sensed hand position toward the cursor position was less in the adaptation condition than in the control condition. Conversely, no such changes were observed for the implicit measure of the bias of sensed hand position, revealing contrasting effects of adaptation on the explicit and implicit measures. These results suggest that biases of explicitly sensed hand position reflect sensory coupling of neural representations that are altered by visuomotor adaptation. In contrast, biases of implicitly sensed hand position reflect sensory coupling of neural representations that are unaffected by adaptation.

Perceptual attraction in tool use: evidence for a reliability-based weighting mechanism

Humans are well able to operate tools whereby their hand movement is linked, via a kinematic transformation, to a spatially distant object moving in a separate plane of motion. An everyday example is controlling a cursor on a computer monitor. Despite these separate reference frames, the perceived positions of the hand and the object were found to be biased toward each other. We propose that this perceptual attraction is based on the principles by which the brain integrates redundant sensory information of single objects or events, known as optimal multisensory integration. That is, 1) sensory information about the hand and the tool are weighted according to their relative reliability (i.e., inverse variances), and 2) the unisensory reliabilities sum up in the integrated estimate. We assessed whether perceptual attraction is consistent with optimal multisensory integration model predictions. We used a cursor-control tool-use task in which we manipulated the relative reliability of the unisensory hand and cursor position estimates. The perceptual biases shifted according to these relative reliabilities, with an additional bias due to contextual factors that were present in experiment 1 but not in experiment 2 The biased position judgments' variances were, however, systematically larger than the predicted optimal variances. Our findings suggest that the perceptual attraction in tool use results from a reliability-based weighting mechanism similar to optimal multisensory integration, but that certain boundary conditions for optimality might not be satisfied.NEW & NOTEWORTHY Kinematic tool use is associated with a perceptual attraction between the spatially separated hand and the effective part of the tool. We provide a formal account for this phenomenon, thereby showing that the process behind it is similar to optimal integration of sensory information relating to single objects.

Are Effects of Action on Perception Real? Evidence from Transformed Movements

It has been argued that several reported non-visual influences on perception cannot be truly perceptual. If they were, they should affect the perception of target objects and reference objects used to express perceptual judgments, and thus cancel each other out. This reasoning presumes that non-visual manipulations impact target objects and comparison objects equally. In the present study we show that equalizing a body-related manipulation between target objects and reference objects essentially abolishes the impact of that manipulation so as it should do when that manipulation actually altered perception. Moreover, the manipulation has an impact on judgements when applied to only the target object but not to the reference object, and that impact reverses when only applied to the reference object but not to the target object. A perceptual explanation predicts this reversal, whereas explanations in terms of post-perceptual response biases or demand effects do not. Altogether these results suggest that body-related influences on perception cannot as a whole be attributed to extra-perceptual factors.

Effects of Reliability and Global Context on Explicit and Implicit Measures of Sensed Hand Position in Cursor-Control Tasks

In a cursor-control task in which the motion of the cursor is rotated randomly relative to the movement of the hand, the sensed directions of hand and cursor are mutually biased. In our previous study, we used implicit and explicit measures of the bias of sensed hand direction toward the direction of the cursor and found different characteristics. The present study serves to explore further differences and commonalities of these measures. In Experiment 1, we examined the effects of different relative reliabilities of visual and proprioceptive information on the explicitly and implicitly assessed bias of sensed hand direction. In two conditions, participants made an aiming movement and returned to the start position immediately or after a delay of 6 s during which the cursor was no longer visible. The unimodal proprioceptive information on final hand position in the delayed condition served to increase its relative reliability. As a result, the bias of sensed hand direction toward the direction of the cursor was reduced for the explicit measure, with a complementary increase of the bias of sensed cursor direction, but unchanged for the implicit measure. In Experiment 2, we examined the influence of global context, specifically of the across-trial sequence of judgments of hand and cursor direction. Both explicitly and implicitly assessed biases of sensed hand direction did not significantly differ between the alternated condition (trial-to-trial alternations of judgments of hand and cursor direction) and the blocked condition (judgments of hand or cursor directions in all trials). They both substantially decreased from the alternated to the randomized condition (random sequence of judgments of hand and cursor direction), without a complementary increase of the bias of sensed cursor direction. We conclude that our explicit and implicit measures are equally sensitive to variations of coupling strength as induced by the variation of global context in Experiment 2, but are differently sensitive to variations of the relative reliabilities as induced by our additional unimodal proprioceptive information in Experiment 1.