Extrapolation of vertical target motion through a brief visual occlusion

Anticipatory smooth pursuit eye movements scale with the probability of visual motion: The role of target speed and acceleration

Sensory-motor systems can extract statistical regularities in dynamic uncertain environments, enabling quicker responses and anticipatory behavior for expected events. Anticipatory smooth pursuit eye movements (aSP) have been observed in primates when the temporal and kinematic properties of a forthcoming visual moving target are fully or partially predictable. To investigate the nature of the internal model of target kinematics underlying aSP, we tested the effect of varying the target kinematics and its predictability. Participants tracked a small visual target in a constant direction with either constant, accelerating, or decelerating speed. Across experimental blocks, we manipulated the probability of each kinematic condition varying either speed or acceleration across trials; with either one kinematic condition (providing certainty) or with a mixture of conditions with a fixed probability within a block. We show that aSP is robustly modulated by target kinematics. With constant-velocity targets, aSP velocity scales linearly with target velocity in blocked sessions, and matches the probability-weighted average in the mixture sessions. Predictable target acceleration does also have an influence on aSP, suggesting that the internal model of motion that drives anticipation contains some information about the changing target kinematics, beyond the initial target speed. However, there is a large variability across participants in the precision and consistency with which this information is taken into account to control anticipatory behavior.

A comparative analysis of perceptual noise in lateral and depth motion: Evidence from eye tracking

The characterization of how precisely we perceive visual speed has traditionally relied on psychophysical judgments in discrimination tasks. Such tasks are often considered laborious and susceptible to biases, particularly without the involvement of highly trained participants. Additionally, thresholds for motion-in-depth perception are frequently reported as higher compared to lateral motion, a discrepancy that contrasts with everyday visuomotor tasks. In this research, we rely on a smooth pursuit model, based on a Kalman filter, to quantify speed observational uncertainties. This model allows us to distinguish between additive and multiplicative noise across three conditions of motion dynamics within a virtual reality setting: random walk, linear motion, and nonlinear motion, incorporating both lateral and depth motion components. We aim to assess tracking performance and perceptual uncertainties for lateral versus motion-in-depth. In alignment with prior research, our results indicate diminished performance for depth motion in the random walk condition, characterized by unpredictable positioning. However, when velocity information is available and facilitates predictions of future positions, perceptual uncertainties become more consistent between lateral and in-depth motion. This consistency is particularly noticeable within ranges where retinal speeds overlap between these two dimensions. Significantly, additive noise emerges as the primary source of uncertainty, largely exceeding multiplicative noise. This predominance of additive noise is consistent with computational accounts of visual motion. Our study challenges earlier beliefs of marked differences in processing lateral versus in-depth motions, suggesting similar levels of perceptual uncertainty and underscoring the significant role of additive noise.

Target interception in virtual reality is better for natural versus unnatural trajectory shapes and orientations

Human performance in perceptual and visuomotor tasks is enhanced when stimulus motion follows the laws of gravitational physics, including acceleration consistent with Earth's gravity, g. Here we used a manual interception task in virtual reality to investigate the effects of trajectory shape and orientation on interception timing and accuracy. Participants punched to intercept a ball moving along one of four trajectories that varied in shape (parabola or tent) and orientation (upright or inverted). We also varied the location of visual fixation such that trajectories fell entirely within the lower or upper visual field. Reaction times were faster for more natural shapes and orientations, regardless of visual field. Overall accuracy was poorer and movement time was longer for the inverted tent condition than the other three conditions, perhaps because it was imperfectly reminiscent of a bouncing ball. A detailed analysis of spatial errors revealed that interception endpoints were more likely to fall along the path of the final trajectory in upright vs. inverted conditions, suggesting stronger expectations regarding the final trajectory direction for these conditions. Taken together, these results suggest that the naturalness of the shape and orientation of a trajectory contributes to performance in a virtual interception task.

Rapid eye and hand responses in an interception task are differentially modulated by context-dependent predictability

When catching a falling ball or avoiding a collision with traffic, humans can quickly generate eye and limb responses to unpredictable changes in their environment. Mechanisms of limb and oculomotor control when responding to sudden changes in the environment have mostly been investigated independently. Here, we investigated eye-hand coordination in a rapid interception task where human participants used a virtual paddle to intercept a moving target. The target moved vertically down a computer screen and could suddenly jump to the left or right. In high-certainty blocks, the target always jumped; in low-certainty blocks, the target only jumped in a portion of the trials. Further, we manipulated response urgency by varying the time of target jumps, with early jumps requiring less urgent responses and late jumps requiring more urgent responses. Our results highlight differential effects of certainty and urgency on eye-hand coordination. Participants initiated both eye and hand responses earlier for high-certainty compared with low-certainty blocks. Hand reaction times decreased and response vigor increased with increasing urgency levels. However, eye reaction times were lowest for medium-urgency levels and eye vigor was unaffected by urgency. Across all trials, we found a weak positive correlation between eye and hand responses. Taken together, these results suggest that the limb and oculomotor systems use similar early sensorimotor processing; however, rapid responses are modulated differentially to attain system-specific sensorimotor goals.

Computational account for the naturalness perception of others' jumping motion based on a vertical projectile motion model

The visual naturalness of a rendered character's motion is an important factor in computer graphics work, and the rendering of jumping motions is no exception to this. However, the computational mechanism that underlies the observer's judgement of the naturalness of a jumping motion has not yet been fully elucidated. We hypothesized that observers would perceive a jumping motion as more natural when the jump trajectory was consistent with the trajectory of a vertical projectile motion based on Earth's gravity. We asked human participants to evaluate the naturalness of point-light jumping motions whose height and duration were modulated. The results showed that the observers' naturalness rating varied with the modulation ratios of the jump height and duration. Interestingly, the ratings were high even when the height and duration differed from the actual jump. To explain this tendency, we constructed computational models that predicted the theoretical trajectory of a jump based on the projectile motion formula and calculated the errors between the theoretical and observed trajectories. The pattern of the errors correlated closely with the participants' ratings. Our results suggest that observers judge the naturalness of observed jumping motion based on the error between observed and predicted jump trajectories.

Motor invariants in action execution and perception

The nervous system is sensitive to statistical regularities of the external world and forms internal models of these regularities to predict environmental dynamics. Given the inherently social nature of human behavior, being capable of building reliable predictive models of others' actions may be essential for successful interaction. While social prediction might seem to be a daunting task, the study of human motor control has accumulated ample evidence that our movements follow a series of kinematic invariants, which can be used by observers to reduce their uncertainty during social exchanges. Here, we provide an overview of the most salient regularities that shape biological motion, examine the role of these invariants in recognizing others' actions, and speculate that anchoring socially-relevant perceptual decisions to such kinematic invariants provides a key computational advantage for inferring conspecifics' goals and intentions.

Estimations of the Passing Height of Approaching Objects

SIGNIFICANCE

Limited optical cues associated with ball flight were inadequate to estimate the vertical passing distance of approaching balls. These results suggest that these optical cues either must be integrated with contextual and kinematic cues or must be of larger amplitude to contribute to estimates of vertical passing distance.

PURPOSE

To intercept or avoid approaching objects, individuals must estimate both when and where the object will arrive. The purpose of this experiment was to determine whether individuals could estimate the vertical passing height of a ball approaching at different linear speeds when vertical angular retinal image velocity and cues for time to contact were minimized.

METHODS

Twenty participants stood 40 feet from a pitching machine that projected tennis balls toward observers at six random speeds from 56 to 80 mph. The flight of the balls was stopped after 9 feet. The actual passing height ranged from about 35 (lowest speed) to 136 cm (highest speed). Observers indicated the height at which they expected the balls to arrive. Overall, the height estimates increased as ball speed increased (means, 121 ± 13 cm [lowest speed] and 131 ± 10 cm [highest speed]). However, only at the higher speeds were the absolute height estimates close to the actual height of the ball. At the higher ball speeds, estimates for participants with some experience in baseball or softball were more accurate (86.4% correct at the highest speed) than estimates for participants with no experience.

CONCLUSIONS

Overall, estimates of vertical passing distance were inaccurate particularly at the lower speeds. Underestimates of vertical drop at lower speeds may have resulted from overestimates of ball speeds. At short exposure durations, optical cues associated with ball flight were inadequate for predictions of vertical passing distance at all speeds for the no-experience group and at lower speeds for the experienced group.

Flexible prediction of opponent motion with internal representation in interception behavior

Skilled interception behavior often relies on accurate predictions of external objects because of a large delay in our sensorimotor systems. To deal with the sensorimotor delay, the brain predicts future states of the target based on the current state available, but it is still debated whether internal representations acquired from prior experience are used as well. Here we estimated the predictive manner by analyzing the response behavior of a pursuer to a sudden directional change of the evasive target, providing strong evidence that prediction of target motion by the pursuer was incompatible with a linear extrapolation based solely on the current state of the target. Moreover, using neural network models, we validated that nonlinear extrapolation as estimated was computationally feasible and useful even against unknown opponents. These results support the use of internal representations in predicting target motion, suggesting the usefulness and versatility of predicting external object motion through internal representations.

Review: Head and Eye Movements and Gaze Tracking in Baseball Batting

SIGNIFICANCE

After a 30-year gap, several studies on head and eye movements and gaze tracking in baseball batting have been performed in the last decade. These baseball studies may lead to training protocols for batting. Here we review these studies and compare the tracking behaviors with those in other sports.Baseball batters are often instructed to "keep your eye on the ball." Until recently, the evidence regarding whether batters follow this instruction and if there are benefits to following this instruction was limited. Baseball batting studies demonstrate that batters tend to move the head more than the eyes in the direction of the ball at least until a saccade occurs. Foveal gaze tracking is often maintained on the ball through the early portion of the pitch, so it can be said that baseball batters do keep the eyes on the ball. While batters place gaze at or near the point of bat-ball contact, the way this is accomplished varies. In some studies, foveal gaze tracking continues late in the pitch trajectory, whereas in other studies, anticipatory saccades occur. The relative advantages of these discrepant gaze strategies on perceptual processing and motor planning speed and accuracy are discussed, and other variables that may influence anticipatory saccades including the predictability of the pitch and the level of batter expertise are described. Further studies involving larger groups with different levels of expertise under game conditions are required to determine which gaze tracking strategies are most beneficial for baseball batting.

Sensorimotor delays in tracking may be compensated by negative feedback control of motion-extrapolated position

Sensorimotor delays dictate that humans act on outdated perceptual information. As a result, continuous manual tracking of an unpredictable target incurs significant response delays. However, no such delays are observed for repeating targets such as the sinusoids. Findings of this kind have led researchers to claim that the nervous system constructs predictive, probabilistic models of the world. However, a more parsimonious explanation is that visual perception of a moving target position is systematically biased by its velocity. The resultant extrapolated position could be compared with the cursor position and the difference canceled by negative feedback control, compensating sensorimotor delays. The current study tested whether a position extrapolation model fit human tracking of sinusoid (predictable) and pseudorandom (less predictable) targets better than the non-biased position control model, Twenty-eight participants tracked these targets and the two computational models were fit to the data at 60 fixed loop delay values (simulating sensorimotor delays). We observed that pseudorandom targets were tracked with a significantly greater phase delay than sinusoid targets. For sinusoid targets, the position extrapolation model simulated tracking results more accurately for loop delays longer than 120 ms, thereby confirming its ability to compensate for sensorimotor delays. However, for pseudorandom targets, this advantage arose only after 300 ms, indicating that velocity information is unlikely to be exploited in this way during the tracking of less predictable targets. We conclude that negative feedback control of position is a parsimonious model for tracking pseudorandom targets and that negative feedback control of extrapolated position is a parsimonious model for tracking sinusoidal targets.

A systematic evaluation of the evidence for perceptual control theory in tracking studies

Perceptual control theory (PCT) proposes that perceptual inputs are controlled to intentional 'reference' states by hierarchical negative feedback control, evidence for which comes from manual tracking experiments in humans. We reviewed these experiments to determine whether tracking is a process of perceptual control, and to assess the state-of-the-evidence for PCT. A systematic literature search was conducted of peer-review journal and book chapters in which tracking data were simulated with a PCT model (13 studies, 53 participants). We report a narrative review of these studies and a qualitative assessment of their methodological quality. We found evidence that individuals track to individual-specific endogenously-specified reference states and act against disturbances, and evidence that hierarchical PCT can simulate complex tracking. PCT's learning algorithm, reorganization, was not modelled. Limitations exist in the range of tracking conditions under which the PCT model has been tested. Future PCT research should apply the PCT methodology to identify control variables in real-world tasks and develop hierarchical PCT architectures for goal-oriented robotics to test the plausibility of PCT model-based action control.

Visuomotor Interactions and Perceptual Judgments in Virtual Reality Simulating Different Levels of Gravity

Virtual reality is used to manipulate sensorimotor interactions in a controlled manner. A critical issue is represented by the extent to which virtual scenarios must conform to physical realism to allow ecological human–machine interactions. Among the physical constraints, Earth gravity is one of the most pervasive and significant for sensorimotor coordination. However, it is still unclear whether visual perception is sensitive to the level of gravity acting on target motion displayed in virtual reality, given the poor visual discrimination of accelerations. To test gravity sensitivity, we asked participants to hit a virtual ball rolling down an incline and falling in air, and to report whether ball motion was perceived as natural or unnatural. We manipulated the gravity level independently for the motion on the incline and for the motion in air. The ball was always visible during rolling, whereas it was visible or occluded during falling before interception. The scene included several cues allowing metric calibration of visual space and motion. We found that the perception rate of natural motion was significantly higher and less variable when ball kinematics was congruent with Earth gravity during both rolling and falling. Moreover, the timing of target interception was accurate only in this condition. Neither naturalness perception nor interception timing depended significantly on whether the target was visible during free-fall. Even when occluded, free-fall under natural gravity was correctly extrapolated from the preceding, visible phase of rolling motion. Naturalness perception depended on motor performance, in addition to the gravity level. In sum, both motor and perceptual responses were guided by an internal model of Earth gravity effects. We suggest that, in order to enhance perceptual sensitivity to physical realism, virtual reality should involve visual backgrounds with metric cues and closed-loop sensorimotor interactions. This suggestion might be especially relevant for the design of rehabilitation protocols.

Earth-Gravity Congruent Motion Facilitates Ocular Control for Pursuit of Parabolic Trajectories

There is evidence that humans rely on an earth gravity (9.81 m/s²) prior for a series of tasks involving perception and action, the reason being that gravity helps predict future positions of moving objects. Eye-movements in turn are partially guided by predictions about observed motion. Thus, the question arises whether knowledge about gravity is also used to guide eye-movements: If humans rely on a representation of earth gravity for the control of eye movements, earth-gravity-congruent motion should elicit improved visual pursuit. In a pre-registered experiment, we presented participants (n = 10) with parabolic motion governed by six different gravities (-1/0.7/0.85/1/1.15/1.3 g), two initial vertical velocities and two initial horizontal velocities in a 3D environment. Participants were instructed to follow the target with their eyes. We tracked their gaze and computed the visual gain (velocity of the eyes divided by velocity of the target) as proxy for the quality of pursuit. An LMM analysis with gravity condition as fixed effect and intercepts varying per subject showed that the gain was lower for -1 g than for 1 g (by -0.13, SE = 0.005). This model was significantly better than a null model without gravity as fixed effect (p < 0.001), supporting our hypothesis. A comparison of 1 g and the remaining gravity conditions revealed that 1.15 g (by 0.043, SE = 0.005) and 1.3 g (by 0.065, SE = 0.005) were associated with lower gains, while 0.7 g (by 0.054, SE = 0.005) and 0.85 g (by 0.029, SE = 0.005) were associated with higher gains. This model was again significantly better than a null model (p < 0.001), contradicting our hypothesis. Post-hoc analyses reveal that confounds in the 0.7/0.85/1/1.15/1.3 g condition may be responsible for these contradicting results. Despite these discrepancies, our data thus provide some support for the hypothesis that internalized knowledge about earth gravity guides eye movements.

Ocular tracking of occluded ballistic trajectories: Effects of visual context and of target law of motion

In tracking a moving target, the visual context may provide cues for an observer to interpret the causal nature of the target motion and extract features to which the visual system is weakly sensitive, such as target acceleration. This information could be critical when vision of the target is temporarily impeded, requiring visual motion extrapolation processes. Here we investigated how visual context influences ocular tracking of motion either congruent or not with natural gravity. To this end, 28 subjects tracked computer-simulated ballistic trajectories either perturbed in the descending segment with altered gravity effects (0g/2g) or retaining natural-like motion (1g). Shortly after the perturbation (550 ms), targets disappeared for either 450 or 650 ms and became visible again until landing. Target motion occurred with either quasi-realistic pictorial cues or a uniform background, presented in counterbalanced order. We analyzed saccadic and pursuit movements after 0g and 2g target-motion perturbations and for corresponding intervals of unperturbed 1g trajectories, as well as after corresponding occlusions. Moreover, we considered the eye-to-target distance at target reappearance. Tracking parameters differed significantly between scenarios: With a neutral background, eye movements did not depend consistently on target motion, whereas with pictorial background they showed significant dependence, denoting better tracking of accelerated targets. These results suggest that oculomotor control is tuned to realistic properties of the visual scene.

Intuitive physics of gravitational motion as shown by perceptual judgment and prediction-motion tasks

In Experiment 1, we explored participants' perceptual knowledge of vertical fall by presenting them with virtually simulated polystyrene or wooden spheres falling to the ground from about two meters high. Participants rated the perceived naturalness of the motion. Besides the implied mass of the sphere, we manipulated the motion pattern (i.e., uniform acceleration vs. uniform velocity), and the magnitude of acceleration or velocity. Results show that relatively low values of acceleration or velocity were judged as natural for the polystyrene sphere, whereas relatively high values of acceleration or velocity were judged as natural for the wooden sphere. In Experiment 2, the same stimuli of Experiment 1 were used, but the sphere disappeared behind an invisible occluder at some point of its trajectory. Participants were asked to predict the time-to-contact (TTC) of the sphere with the ground by pressing a key at the exact time of impact of the lower edge of the sphere with the floor of the room. Results show that the estimated TTC for the simulated wooden sphere was slightly but consistently smaller than the estimated TTC for the simulated polystyrene sphere. The influence of the implied mass on participants' responses might be the manifestation of two processes, namely an explicit 'heavy-fast, light-slow' heuristic, and/or an implicit, automatic association between mass and falling speed.

Continuously updating one’s predictions underlies successful interception

This paper reviews our understanding of the interception of moving objects. Interception is a demanding task that requires both spatial and temporal precision. The required precision must be achieved on the basis of imprecise and sometimes biased sensory information. We argue that people make precise interceptive movements by continuously adjusting their movements. Initial estimates of how the movement should progress can be quite inaccurate. As the movement evolves, the estimate of how the rest of the movement should progress gradually becomes more reliable as prediction is replaced by sensory information about the progress of the movement. The improvement is particularly important when things do not progress as anticipated. Constantly adjusting one’s estimate of how the movement should progress combines the opportunity to move in a way that one anticipates will best meet the task demands with correcting for any errors in such anticipation. The fact that the ongoing movement might have to be adjusted can be considered when determining how to move, and any systematic anticipation errors can be corrected on the basis of the outcome of earlier actions.

Rolling Motion Along an Incline: Visual Sensitivity to the Relation Between Acceleration and Slope

People easily intercept a ball rolling down an incline, despite its acceleration varies with the slope in a complex manner. Apparently, however, they are poor at detecting anomalies when asked to judge artificial animations of descending motion. Since the perceptual deficiencies have been reported in studies involving a limited visual context, here we tested the hypothesis that judgments of naturalness of rolling motion are consistent with physics when the visual scene incorporates sufficient cues about environmental reference and metric scale, roughly comparable to those present when intercepting a ball. Participants viewed a sphere rolling down an incline located in the median sagittal plane, presented in 3D wide-field virtual reality. In different experiments, either the slope of the plane or the sphere acceleration were changed in arbitrary combinations, resulting in a kinematics that was either consistent or inconsistent with physics. In Experiment 1 (slope adjustment), participants were asked to modify the slope angle until the resulting motion looked natural for a given ball acceleration. In Experiment 2 (acceleration adjustment), instead, they were asked to modify the acceleration until the motion on a given slope looked natural. No feedback about performance was provided. For both experiments, we found that participants were rather accurate at finding the match between slope angle and ball acceleration congruent with physics, but there was a systematic effect of the initial conditions: accuracy was higher when the participants started the exploration from the combination of slope and acceleration corresponding to the congruent conditions than when they started far away from the congruent conditions. In Experiment 3, participants modified the slope angle based on an adaptive staircase, but the target never coincided with the starting condition. Here we found a generally accurate performance, irrespective of the target slope. We suggest that, provided the visual scene includes sufficient cues about environmental reference and metric scale, joint processing of slope and acceleration may facilitate the detection of natural motion. Perception of rolling motion may rely on the kind of approximate, probabilistic simulations of Newtonian mechanics that have previously been called into play to explain complex inferences in rich visual scenes.

Mental imagery of gravitational motion

There is considerable evidence that gravitational acceleration is taken into account in the interaction with falling targets through an internal model of Earth gravity. Here we asked whether this internal model is accessed also when target motion is imagined rather than real. In the main experiments, naïve participants grasped an imaginary ball, threw it against the ceiling, and caught it on rebound. In different blocks of trials, they had to imagine that the ball moved under terrestrial gravity (1g condition) or under microgravity (0g) as during a space flight. We measured the speed and timing of the throwing and catching actions, and plotted ball flight duration versus throwing speed. Best-fitting duration-speed curves estimate the laws of ball motion implicit in the participant's performance. Surprisingly, we found duration-speed curves compatible with 0g for both the imaginary 0g condition and the imaginary 1g condition, despite the familiarity with Earth gravity effects and the added realism of performing the throwing and catching actions. In a control experiment, naïve participants were asked to throw the imaginary ball vertically upwards at different heights, without hitting the ceiling, and to catch it on its way down. All participants overestimated ball flight durations relative to the durations predicted by the effects of Earth gravity. Overall, the results indicate that mental imagery of motion does not have access to the internal model of Earth gravity, but resorts to a simulation of visual motion. Because visual processing of accelerating/decelerating motion is poor, visual imagery of motion at constant speed or slowly varying speed appears to be the preferred mode to perform the tasks.

Differential contributions to the interception of occluded ballistic trajectories by the temporoparietal junction, area hMT/V5+, and the intraparietal cortex

The ability to catch objects when transiently occluded from view suggests their motion can be extrapolated. Intraparietal cortex (IPS) plays a major role in this process along with other brain structures, depending on the task. For example, interception of objects under Earth's gravity effects may depend on time-to-contact predictions derived from integration of visual signals processed by hMT/V5+ with a priori knowledge of gravity residing in the temporoparietal junction (TPJ). To investigate this issue further, we disrupted TPJ, hMT/V5+, and IPS activities with transcranial magnetic stimulation (TMS) while subjects intercepted computer-simulated projectile trajectories perturbed randomly with either hypo- or hypergravity effects. In experiment 1, trajectories were occluded either 750 or 1,250 ms before landing. Three subject groups underwent triple-pulse TMS (tpTMS, 3 pulses at 10 Hz) on one target area (TPJ | hMT/V5+ | IPS) and on the vertex (control site), timed at either trajectory perturbation or occlusion. In experiment 2, trajectories were entirely visible and participants received tpTMS on TPJ and hMT/V5+ with same timing as experiment 1 tpTMS of TPJ, hMT/V5+, and IPS affected differently the interceptive timing. TPJ stimulation affected preferentially responses to 1-g motion, hMT/V5+ all response types, and IPS stimulation induced opposite effects on 0-g and 2-g responses, being ineffective on 1-g responses. Only IPS stimulation was effective when applied after target disappearance, implying this area might elaborate memory representations of occluded target motion. Results are compatible with the idea that IPS, TPJ, and hMT/V5+ contribute to distinct aspects of visual motion extrapolation, perhaps through parallel processing.NEW & NOTEWORTHY Visual extrapolation represents a potential neural solution to afford motor interactions with the environment in the face of missing information. We investigated relative contributions by temporoparietal junction (TPJ), hMT/V5+, and intraparietal cortex (IPS), cortical areas potentially involved in these processes. Parallel organization of visual extrapolation processes emerged with respect to the target's motion causal nature: TPJ was primarily involved for visual motion congruent with gravity effects, IPS for arbitrary visual motion, whereas hMT/V5+ contributed at earlier processing stages.

Gravity as a Strong Prior: Implications for Perception and Action

In the future, humans are likely to be exposed to environments with altered gravity conditions, be it only visually (Virtual and Augmented Reality), or visually and bodily (space travel). As visually and bodily perceived gravity as well as an interiorized representation of earth gravity are involved in a series of tasks, such as catching, grasping, body orientation estimation and spatial inferences, humans will need to adapt to these new gravity conditions. Performance under earth gravity discrepant conditions has been shown to be relatively poor, and few studies conducted in gravity adaptation are rather discouraging. Especially in VR on earth, conflicts between bodily and visual gravity cues seem to make a full adaptation to visually perceived earth-discrepant gravities nearly impossible, and even in space, when visual and bodily cues are congruent, adaptation is extremely slow. We invoke a Bayesian framework for gravity related perceptual processes, in which earth gravity holds the status of a so called “strong prior”. As other strong priors, the gravity prior has developed through years and years of experience in an earth gravity environment. For this reason, the reliability of this representation is extremely high and overrules any sensory information to its contrary. While also other factors such as the multisensory nature of gravity perception need to be taken into account, we present the strong prior account as a unifying explanation for empirical results in gravity perception and adaptation to earth-discrepant gravities.

Filling gaps in visual motion for target capture

A remarkable challenge our brain must face constantly when interacting with the environment is represented by ambiguous and, at times, even missing sensory information. This is particularly compelling for visual information, being the main sensory system we rely upon to gather cues about the external world. It is not uncommon, for example, that objects catching our attention may disappear temporarily from view, occluded by visual obstacles in the foreground. Nevertheless, we are often able to keep our gaze on them throughout the occlusion or even catch them on the fly in the face of the transient lack of visual motion information. This implies that the brain can fill the gaps of missing sensory information by extrapolating the object motion through the occlusion. In recent years, much experimental evidence has been accumulated that both perceptual and motor processes exploit visual motion extrapolation mechanisms. Moreover, neurophysiological and neuroimaging studies have identified brain regions potentially involved in the predictive representation of the occluded target motion. Within this framework, ocular pursuit and manual interceptive behavior have proven to be useful experimental models for investigating visual extrapolation mechanisms. Studies in these fields have pointed out that visual motion extrapolation processes depend on manifold information related to short-term memory representations of the target motion before the occlusion, as well as to longer term representations derived from previous experience with the environment. We will review recent oculomotor and manual interception literature to provide up-to-date views on the neurophysiological underpinnings of visual motion extrapolation.

Familiar trajectories facilitate the interpretation of physical forces when intercepting a moving target

Familiarity with the visual environment affects our expectations about the objects in a scene, aiding in recognition and interaction. Here we tested whether the familiarity with the specific trajectory followed by a moving target facilitates the interpretation of the effects of underlying physical forces. Participants intercepted a target sliding down either an inclined plane or a tautochrone. Gravity accelerated the target by the same amount in both cases, but the inclined plane represented a familiar trajectory whereas the tautochrone was unfamiliar to the participants. In separate sessions, the gravity field was consistent with either natural gravity or artificial reversed gravity. Target motion was occluded from view over the last segment. We found that the responses in the session with unnatural forces were systematically delayed relative to those with natural forces, but only for the inclined plane. The time shift is consistent with a bias for natural gravity, in so far as it reflects an a priori expectation that a target not affected by natural forces will arrive later than one accelerated downwards by gravity. Instead, we did not find any significant time shift with unnatural forces in the case of the tautochrone. We argue that interception of a moving target relies on the integration of the high-level cue of trajectory familiarity with low-level cues related to target kinematics.

Neural extrapolation of motion for a ball rolling down an inclined plane

It is known that humans tend to misjudge the kinematics of a target rolling down an inclined plane. Because visuomotor responses are often more accurate and less prone to perceptual illusions than cognitive judgments, we asked the question of how rolling motion is extrapolated for manual interception or drawing tasks. In three experiments a ball rolled down an incline with kinematics that differed as a function of the starting position (4 different positions) and slope (30°, 45° or 60°). In Experiment 1, participants had to punch the ball as it fell off the incline. In Experiment 2, the ball rolled down the incline but was stopped at the end; participants were asked to imagine that the ball kept moving and to punch it. In Experiment 3, the ball rolled down the incline and was stopped at the end; participants were asked to draw with the hand in air the trajectory that would be described by the ball if it kept moving. We found that performance was most accurate when motion of the ball was visible until interception and haptic feedback of hand-ball contact was available (Experiment 1). However, even when participants punched an imaginary moving ball (Experiment 2) or drew in air the imaginary trajectory (Experiment 3), they were able to extrapolate to some extent global aspects of the target motion, including its path, speed and arrival time. We argue that the path and kinematics of a ball rolling down an incline can be extrapolated surprisingly well by the brain using both visual information and internal models of target motion.

Catching what we can't see: manual interception of occluded fly-ball trajectories

Control of interceptive actions may involve fine interplay between feedback-based and predictive mechanisms. These processes rely heavily on target motion information available when the target is visible. However, short-term visual memory signals as well as implicit knowledge about the environment may also contribute to elaborate a predictive representation of the target trajectory, especially when visual feedback is partially unavailable because other objects occlude the visual target. To determine how different processes and information sources are integrated in the control of the interceptive action, we manipulated a computer-generated visual environment representing a baseball game. Twenty-four subjects intercepted fly-ball trajectories by moving a mouse cursor and by indicating the interception with a button press. In two separate sessions, fly-ball trajectories were either fully visible or occluded for 750, 1000 or 1250 ms before ball landing. Natural ball motion was perturbed during the descending trajectory with effects of either weightlessness (0 g) or increased gravity (2 g) at times such that, for occluded trajectories, 500 ms of perturbed motion were visible before ball disappearance. To examine the contribution of previous visual experience with the perturbed trajectories to the interception of invisible targets, the order of visible and occluded sessions was permuted among subjects. Under these experimental conditions, we showed that, with fully visible targets, subjects combined servo-control and predictive strategies. Instead, when intercepting occluded targets, subjects relied mostly on predictive mechanisms based, however, on different type of information depending on previous visual experience. In fact, subjects without prior experience of the perturbed trajectories showed interceptive errors consistent with predictive estimates of the ball trajectory based on a-priori knowledge of gravity. Conversely, the interceptive responses of subjects previously exposed to fully visible trajectories were compatible with the fact that implicit knowledge of the perturbed motion was also taken into account for the extrapolation of occluded trajectories.

To know or not to know: influence of explicit advance knowledge of occlusion on interceptive actions

This study examined how explicit advance knowledge might influence adaptive behavior to visual occlusions. Catching performance and kinematics of good ball catchers were compared between no, early and late occlusion trials. Discrete visual occlusions of 400 ms, occurring early or late in the ball's approach trajectory, were randomly interspersed between no occlusion trials. In one condition, the presence and type of occlusion were announced a priori (expected), whereas in another condition no such information was provided (unexpected). Expectation of occlusion resulted in an adapted limb transport and increased grasping time, whereas in the unexpected condition a higher peak of wrist velocity was evident for all occlusion conditions. The observed different adaptations cannot be explained by trial-by-trial adaptations alone and instead provide evidence for the influence of explicit advance knowledge in the motor response of interceptive actions.

The Effect of Body Posture on Long-Range Time-to-Contact Estimation

On Earth, gravity accelerates freely moving objects downward, whereas upward-moving objects are being decelerated. Do humans take internalised knowledge of gravity into account when estimating time-to-contact (TTC, the time remaining before the moving object reaches the observer)? To answer this question, we created a motion-prediction task in which participants saw the initial part of an object's trajectory moving on a collision course prior to an occlusion. Observers had to judge when the object would make contact with them. The visual scene was presented with a head-mounted display. Participants lay either supine (looking up) or prone (looking down), suggestive of the ball either rising up or falling down toward them. Results showed that body posture had a significant effect on time-to-contact estimation, but only when occlusion times were long (2.5 s). The effect was also rather small. This lack of immediacy in the posture effect suggests that TTC estimation is initially robust toward the effect of gravity, which comes to bear only as more time is allowed for post-processing of the visual information.