Egocentric and allocentric reference frames for catching a falling object

Human estimates of descending objects' motion are more accurate than those of ascending objects regardless of gravity information

Humans can accurately estimate and track object motion, even if it accelerates. Research shows that humans exhibit superior estimation and tracking performance for descending (falling) than ascending (rising) objects. Previous studies presented ascending and descending targets along the gravitational and body axes in an upright posture. Thus, it is unclear whether humans rely on congruent information between the direction of the target motion and gravity or the direction of the target motion and longitudinal body axes. Two experiments were conducted to explore these possibilities. In Experiment 1, participants estimated the arrival time at a goal for both upward and downward motion of targets along the longitudinal body axis in the upright (both axes of target motion and gravity congruent) and supine (both axes incongruent) postures. In Experiment 2, smooth pursuit eye movements were assessed while tracking both targets in the same postures. Arrival time estimation and smooth pursuit eye movement performance were consistently more accurate for downward target motion than for upward motion, irrespective of posture. These findings suggest that the visual experience of seeing an object moving along an observer's leg side in everyday life may influence the ability to accurately estimate and track the descending object's motion.

Contributions of Body-Orientation to Mental Ball Dropping Task During Out-of-Body Experiences

Out-of-body experiences (OBEs) provide fascinating insights into our understanding of bodily self-consciousness and the workings of the brain. Studies that examined individuals with brain lesions reported that OBEs are generally characterized by participants experiencing themselves outside their physical body (i.e., disembodied feeling) (Blanke and Arzy, 2005). Based on such a characterization, it has been shown that it is possible to create virtual OBEs in immersive virtual environments (Ehrsson, 2007; Ionta et al., 2011b; Bourdin et al., 2017). However, the extent to which body-orientation influences virtual OBEs is not well-understood. Thus, in the present study, 30 participants (within group design) experienced a full-body ownership illusion (synchronous visuo-tactile stimulation only) induced with a gender-matched full-body virtual avatar seen from the first-person perspective (1PP). At the beginning of the experiment, participants performed a mental ball dropping (MBD) task, seen from the location of their virtual avatar, to provide a baseline measurement. After this, a full-body ownership illusion (embodiment phase) was induced in all participants. This was followed by the virtual OBE illusion phase of the experiment (disembodiment phase) in which the first-person viewpoint was switched to a third-person perspective (3PP), and participants' disembodied viewpoint was gradually raised to 14 m above the virtual avatar, from which altitude they repeated the MBD task. During the experiment, this procedure was conducted twice, and the participants were allocated first to the supine or the standing body position at random. Results of the MBD task showed that the participants experienced increased MBD durations during the supine condition compared to the standing condition. Furthermore, although the findings from the subjective reports confirmed the previous findings of virtual OBEs, no significant difference between the two postures was found for body ownership. Taken together, the findings of the current study make further contributions to our understanding of both the vestibular system and time perception during OBEs.

Watching the Effects of Gravity. Vestibular Cortex and the Neural Representation of "Visual" Gravity

Gravity is a physical constraint all terrestrial species have adapted to through evolution. Indeed, gravity effects are taken into account in many forms of interaction with the environment, from the seemingly simple task of maintaining balance to the complex motor skills performed by athletes and dancers. Graviceptors, primarily located in the vestibular otolith organs, feed the Central Nervous System with information related to the gravity acceleration vector. This information is integrated with signals from semicircular canals, vision, and proprioception in an ensemble of interconnected brain areas, including the vestibular nuclei, cerebellum, thalamus, insula, retroinsula, parietal operculum, and temporo-parietal junction, in the so-called vestibular network. Classical views consider this stage of multisensory integration as instrumental to sort out conflicting and/or ambiguous information from the incoming sensory signals. However, there is compelling evidence that it also contributes to an internal representation of gravity effects based on prior experience with the environment. This a priori knowledge could be engaged by various types of information, including sensory signals like the visual ones, which lack a direct correspondence with physical gravity. Indeed, the retinal accelerations elicited by gravitational motion in a visual scene are not invariant, but scale with viewing distance. Moreover, the "visual" gravity vector may not be aligned with physical gravity, as when we watch a scene on a tilted monitor or in weightlessness. This review will discuss experimental evidence from behavioral, neuroimaging (connectomics, fMRI, TMS), and patients' studies, supporting the idea that the internal model estimating the effects of gravity on visual objects is constructed by transforming the vestibular estimates of physical gravity, which are computed in the brainstem and cerebellum, into internalized estimates of virtual gravity, stored in the vestibular cortex. The integration of the internal model of gravity with visual and non-visual signals would take place at multiple levels in the cortex and might involve recurrent connections between early visual areas engaged in the analysis of spatio-temporal features of the visual stimuli and higher visual areas in temporo-parietal-insular regions.

Body orientation contributes to modelling the effects of gravity for target interception in humans

KEY POINTS

It is known that interception of targets accelerated by gravity involves internal models coupled with visual signals. Non-visual signals related to head and body orientation relative to gravity may also contribute, although their role is poorly understood. In a novel experiment, we asked pitched observers to hit a virtual target approaching with an acceleration that was either coherent or incoherent with their pitch-tilt. Initially, the timing errors were large and independent of the coherence between target acceleration and observer's pitch. With practice, however, the timing errors became substantially smaller in the coherent conditions. The results show that information about head and body orientation can contribute to modelling the effects of gravity on a moving target. Orientation cues from vestibular and somatosensory signals might be integrated with visual signals in the vestibular cortex, where the internal model of gravity is assumed to be encoded.

ABSTRACT

Interception of moving targets relies on visual signals and internal models. Less is known about the additional contribution of non-visual cues about head and body orientation relative to gravity. We took advantage of Galileo's law of motion along an incline to demonstrate the effects of vestibular and somatosensory cues about head and body orientation on interception timing. Participants were asked to hit a ball rolling in a gutter towards the eyes, resulting in image expansion. The scene was presented in a head-mounted display, without any visual information about gravity direction. In separate blocks of trials participants were pitched backwards by 20° or 60°, whereas ball acceleration was randomized across trials so as to be compatible with rolling down a slope of 20° or 60°. Initially, the timing errors were large, independently of the coherence between ball acceleration and pitch angle, consistent with responses based exclusively on visual information because visual stimuli were identical at both tilts. At the end of the experiment, however, the timing errors were systematically smaller in the coherent conditions than the incoherent ones. Moreover, the responses were significantly (P = 0.007) earlier when participants were pitched by 60° than when they were pitched by 20°. Therefore, practice with the task led to incorporation of information about head and body orientation relative to gravity for response timing. Instead, posture did not affect response timing in a control experiment in which participants hit a static target in synchrony with the last of a predictable series of stationary audiovisual stimuli.

A novel dissociation between representational momentum and representational gravity through response modality

When people are required to indicate the vanishing location of a moving object, systematic biases forward, in the direction of motion, and downward, in the direction of gravity, are usually found. Both these displacements, called representational momentum and representational gravity, respectively, are thought to reflect anticipatory internal mechanisms aiming to overcome neural delays in the perception of motion. We challenge this view. There may not be such a single mechanism. Although both representational momentum and representational gravity follow a specific time-course, compatible with an anticipation of the object's dynamics, they do not seem to be commensurable with each other, as they are differentially modulated by relevant variables, such as eye movements and strength of motion signals. We found separate response components, one related to overt motor localization behaviour and one limited to purely perceptual judgement. Representational momentum emerged only for the motor localization task, revealing a motor overshoot. In contrast, representational gravity was mostly evident for spatial perceptual judgements. We interpret the results in support of a partial dissociation in the mechanisms that give rise to representational momentum and representational gravity, with the former but not the latter strongly modulated by the enrolment of the motor system.

Gravity in the Brain as a Reference for Space and Time Perception

Moving and interacting with the environment require a reference for orientation and a scale for calibration in space and time. There is a wide variety of environmental clues and calibrated frames at different locales, but the reference of gravity is ubiquitous on Earth. The pull of gravity on static objects provides a plummet which, together with the horizontal plane, defines a three-dimensional Cartesian frame for visual images. On the other hand, the gravitational acceleration of falling objects can provide a time-stamp on events, because the motion duration of an object accelerated by gravity over a given path is fixed. Indeed, since ancient times, man has been using plumb bobs for spatial surveying, and water clocks or pendulum clocks for time keeping. Here we review behavioral evidence in favor of the hypothesis that the brain is endowed with mechanisms that exploit the presence of gravity to estimate the spatial orientation and the passage of time. Several visual and non-visual (vestibular, haptic, visceral) cues are merged to estimate the orientation of the visual vertical. However, the relative weight of each cue is not fixed, but depends on the specific task. Next, we show that an internal model of the effects of gravity is combined with multisensory signals to time the interception of falling objects, to time the passage through spatial landmarks during virtual navigation, to assess the duration of a gravitational motion, and to judge the naturalness of periodic motion under gravity.

The embodied dynamics of perceptual causality: a slippery slope?

In Michotte's launching displays, while the launcher (object A) seems to move autonomously, the target (object B) seems to be displaced passively. However, the impression of A actively launching B does not persist beyond a certain distance identified as the "radius of action" of A over B. If the target keeps moving beyond the radius of action, it loses its passivity and seems to move autonomously. Here, we manipulated implied friction by drawing (or not) a surface upon which A and B are traveling, and by varying the inclination of this surface in screen- and earth-centered reference frames. Among 72 participants (n = 52 in Experiment 1; n = 20 in Experiment 2), we show that both physical embodiment of the event (looking straight ahead at a screen displaying the event on a vertical plane vs. looking downwards at the event displayed on a horizontal plane) and contextual information (objects moving along a depicted surface or in isolation) affect interpretation of the event and modulate the radius of action of the launcher. Using classical mechanics equations, we show that representational consistency of friction from radius of action responses emphasizes the embodied nature of frictional force in our cognitive architecture.

Familiar trajectories facilitate the interpretation of physical forces when intercepting a moving target

Familiarity with the visual environment affects our expectations about the objects in a scene, aiding in recognition and interaction. Here we tested whether the familiarity with the specific trajectory followed by a moving target facilitates the interpretation of the effects of underlying physical forces. Participants intercepted a target sliding down either an inclined plane or a tautochrone. Gravity accelerated the target by the same amount in both cases, but the inclined plane represented a familiar trajectory whereas the tautochrone was unfamiliar to the participants. In separate sessions, the gravity field was consistent with either natural gravity or artificial reversed gravity. Target motion was occluded from view over the last segment. We found that the responses in the session with unnatural forces were systematically delayed relative to those with natural forces, but only for the inclined plane. The time shift is consistent with a bias for natural gravity, in so far as it reflects an a priori expectation that a target not affected by natural forces will arrive later than one accelerated downwards by gravity. Instead, we did not find any significant time shift with unnatural forces in the case of the tautochrone. We argue that interception of a moving target relies on the integration of the high-level cue of trajectory familiarity with low-level cues related to target kinematics.

To pass or not to pass: more a question of body orientation than visual cues

This study investigated the influence of pitch body tilt on judging the possibility of passing under high obstacles in the presence of an illusory horizontal self-motion. Seated subjects tilted at various body orientations were asked to estimate the possibility of passing under a projected bar (i.e., a parking barrier), while imagining a forward whole-body displacement normal to gravity. This task was performed under two visual conditions, providing either no visual surroundings or a translational horizontal optic flow that stopped just before the barrier appeared. The results showed a main overestimation of the possibility of passing under the bar in both cases and most importantly revealed a strong influence of body orientation despite the visual specification of horizontal self-motion by optic flow (i.e., both visual conditions yielded a comparable body tilt effect). Specifically, the subjective passability was proportionally deviated towards the body tilt by 46% of its magnitude when facing a horizontal optic flow and 43% without visual surroundings. This suggests that the egocentric attraction exerted by body tilt when referring the subjective passability to horizontal self-motion still persists even when anchoring horizontally related visual cues are displayed. These findings are discussed in terms of interaction between spatial references. The link between the reliability of available sensory inputs and the weight attributed to each reference is also addressed.

Energy-related optimal control accounts for gravitational load: comparing shoulder, elbow, and wrist rotations

We permanently deal with gravity force. Experimental evidences revealed that moving against gravity strongly differs from moving along the gravity vector. This directional asymmetry has been attributed to an optimal planning process that optimizes gravity force effects to minimize energy. Yet, only few studies have considered the case of vertical movements in the context of optimal control. What kind of cost is better suited to explain kinematic patterns in the vertical plane? Here, we aimed to understand further how the central nervous system (CNS) plans and controls vertical arm movements. Our reasoning was the following: if the CNS optimizes gravity mechanical effects on the moving limbs, kinematic patterns should change according to the direction and the magnitude of the gravity torque being encountered in the motion. Ten subjects carried out single-joint movements, i.e., rotation around the shoulder (whole arm), elbow (forearm), and wrist (hand) joints, in the vertical plane. Joint kinematics were analyzed and compared with various theoretical optimal model predictions (minimum absolute work-jerk, jerk, torque change, and variance). We found both direction-dependent and joint-dependent variations in several kinematic parameters. Notably, directional asymmetries decreased according to a proximodistal gradient. Numerical simulations revealed that our experimental findings could be attributed to an optimal motor planning (minimum absolute work-jerk) that integrates the direction and the magnitude of gravity torque and minimizes the absolute work of forces (energy-related cost) around each joint. Present results support the general idea that the CNS implements optimal solutions according to the dynamic context of the action.

When up is down in 0g: how gravity sensing affects the timing of interceptive actions

Humans are known to regulate the timing of interceptive actions by modeling, in a simplified way, Newtonian mechanics. Specifically, when intercepting an approaching ball, humans trigger their movements a bit earlier when the target arrives from above than from below. This bias occurs regardless of the ball's true kinetics, and thus appears to reflect an a priori expectation that a downward moving object will accelerate. We postulate that gravito-inertial information is used to tune visuomotor responses to match the target's most likely acceleration. Here we used the peculiar conditions of parabolic flight--where gravity's effects change every 20 s--to test this hypothesis. We found a striking reversal in the timing of interceptive responses performed in weightlessness compared with trials performed on ground, indicating a role of gravity sensing in the tuning of this response. Parallels between these observations and the properties of otolith receptors suggest that vestibular signals themselves might plausibly provide the critical input. Thus, in addition to its acknowledged importance for postural control, gaze stabilization, and spatial navigation, we propose that detecting the direction of gravity's pull plays a role in coordinating quick reactions intended to intercept a fast-moving visual target.

Perceived object stability depends on multisensory estimates of gravity

Background

How does the brain estimate object stability? Objects fall over when the gravity-projected centre-of-mass lies outside the point or area of support. To estimate an object's stability visually, the brain must integrate information across the shape and compare its orientation to gravity. When observers lie on their sides, gravity is perceived as tilted toward body orientation, consistent with a representation of gravity derived from multisensory information. We exploited this to test whether vestibular and kinesthetic information affect this visual task or whether the brain estimates object stability solely from visual information.

Methodology/Principal Findings

In three body orientations, participants viewed images of objects close to a table edge. We measured the critical angle at which each object appeared equally likely to fall over or right itself. Perceived gravity was measured using the subjective visual vertical. The results show that the perceived critical angle was significantly biased in the same direction as the subjective visual vertical (i.e., towards the multisensory estimate of gravity).

Conclusions/Significance

Our results rule out a general explanation that the brain depends solely on visual heuristics and assumptions about object stability. Instead, they suggest that multisensory estimates of gravity govern the perceived stability of objects, resulting in objects appearing more stable than they are when the head is tilted in the same direction in which they fall.

The Effect of Body Posture on Long-Range Time-to-Contact Estimation

On Earth, gravity accelerates freely moving objects downward, whereas upward-moving objects are being decelerated. Do humans take internalised knowledge of gravity into account when estimating time-to-contact (TTC, the time remaining before the moving object reaches the observer)? To answer this question, we created a motion-prediction task in which participants saw the initial part of an object's trajectory moving on a collision course prior to an occlusion. Observers had to judge when the object would make contact with them. The visual scene was presented with a head-mounted display. Participants lay either supine (looking up) or prone (looking down), suggestive of the ball either rising up or falling down toward them. Results showed that body posture had a significant effect on time-to-contact estimation, but only when occlusion times were long (2.5 s). The effect was also rather small. This lack of immediacy in the posture effect suggests that TTC estimation is initially robust toward the effect of gravity, which comes to bear only as more time is allowed for post-processing of the visual information.