Grasping adjustments to haptic, visual, and visuo-haptic object perturbations are contingent on the sensory modality

Reach Corrections Toward Moving Objects are Faster Than Reach Corrections Toward Instantaneously Switching Targets

Visually guided reaching is a common motor behavior that engages subcortical circuits to mediate rapid corrections. Although these neural mechanisms have evolved for interacting with the physical world, they are often studied in the context of reaching toward virtual targets on a screen. These targets often change position by disappearing from one place reappearing in another instantaneously. In this study, we instructed participants to perform rapid reaches to physical objects that changed position in different ways. In one condition, the objects moved very rapidly from one place to another. In the other condition, illuminated targets instantaneously switched position by being extinguished in one position and illuminating in another. Participants were consistently faster in correcting their reach trajectories when the object moved continuously.

The integration of action-oriented multisensory information from target and limb within the movement planning and execution

The planning and execution of a grasping or reaching movement toward targets we sense with the other hand requires integrating multiple sources of sensory information about the limb performing the movement and the target of the action. In the last two decades, several sensory and motor control theories have thoroughly described how this multisensory-motor integration process occurs. However, even though these theories were very influential in their respective field, they lack a clear, unified vision of how target-related and movement-related multisensory information integrates within the action planning and execution phases. This brief review aims to summarize the most influential theories in multisensory integration and sensory-motor control by underscoring their critical points and hidden connections, providing new ideas on the multisensory-motor integration process. Throughout the review, I wll propose an alternative view of how the multisensory integration process unfolds along the action planning and execution and I will make several connections with the existent multisensory-motor control theories.

Visual uncertainty unveils the distinct role of haptic cues in multisensory grasping

Human multisensory grasping movements (i.e., seeing and feeling a handheld object while grasping it with the contralateral hand) are superior to movements guided by each separate modality. This multisensory advantage might be driven by the integration of vision with either the haptic position only or with both position and size cues. To contrast these two hypotheses, we manipulated visual uncertainty (central vs. peripheral vision) and the availability of haptic cues during multisensory grasping. We showed a multisensory benefit irrespective of the degree of visual uncertainty suggesting that the integration process involved in multisensory grasping can be flexibly modulated by the contribution of each modality. Increasing visual uncertainty revealed the role of the distinct haptic cues. The haptic position cue was sufficient to promote multisensory benefits evidenced by faster actions with smaller grip apertures, whereas the haptic size was fundamental in fine-tuning the grip aperture scaling. These results support the hypothesis that, in multisensory grasping, vision is integrated with all haptic cues, with the haptic position cue playing the key part. Our findings highlight the important role of non-visual sensory inputs in sensorimotor control and hint at the potential contributions of the haptic modality in developing and maintaining visuomotor functions.Significance statementThe longstanding view that vision is considered the primary sense we rely on to guide grasping movements relegates the equally important haptic inputs, such as touch and proprioception, to a secondary role. Here we show that by increasing visual uncertainty during visuo-haptic grasping, the central nervous system exploits distinct haptic inputs about the object position and size to optimize grasping performance. Specifically, we demonstrate that haptic inputs about the object position are fundamental to support vision in enhancing grasping performance, whereas haptic size inputs can further refine hand shaping. Our results provide strong evidence that non-visual inputs serve an important, previously under-appreciated, functional role in grasping.

Somesthetic, Visual, and Auditory Feedback and Their Interactions Applied to Upper Limb Neurorehabilitation Technology: A Narrative Review to Facilitate Contextualization of Knowledge

Reduced hand dexterity is a common component of sensorimotor impairments for individuals after stroke. To improve hand function, innovative rehabilitation interventions are constantly developed and tested. In this context, technology-based interventions for hand rehabilitation have been emerging rapidly. This paper offers an overview of basic knowledge on post lesion plasticity and sensorimotor integration processes in the context of augmented feedback and new rehabilitation technologies, in particular virtual reality and soft robotic gloves. We also discuss some factors to consider related to the incorporation of augmented feedback in the development of technology-based interventions in rehabilitation. This includes factors related to feedback delivery parameter design, task complexity and heterogeneity of sensory deficits in individuals affected by a stroke. In spite of the current limitations in our understanding of the mechanisms involved when using new rehabilitation technologies, the multimodal augmented feedback approach appears promising and may provide meaningful ways to optimize recovery after stroke. Moving forward, we argue that comparative studies allowing stratification of the augmented feedback delivery parameters based upon different biomarkers, lesion characteristics or impairments should be advocated (e.g., injured hemisphere, lesion location, lesion volume, sensorimotor impairments). Ultimately, we envision that treatment design should combine augmented feedback of multiple modalities, carefully adapted to the specific condition of the individuals affected by a stroke and that evolves along with recovery. This would better align with the new trend in stroke rehabilitation which challenges the popular idea of the existence of an ultimate good-for-all intervention.

Using Immersive Virtual Reality to Examine How Visual and Tactile Cues Drive the Material-Weight Illusion

The material-weight illusion (MWI) demonstrates how our past experience with material and weight can create expectations that influence the perceived heaviness of an object. Here we used mixed-reality to place touch and vision in conflict, to investigate whether the modality through which materials are presented to a lifter could influence the top-down perceptual processes driving the MWI. University students lifted equally-weighted polystyrene, cork and granite cubes whilst viewing computer-generated images of the cubes in virtual reality (VR). This allowed the visual and tactile material cues to be altered, whilst all other object properties were kept constant. Representation of the objects’ material in VR was manipulated to create four sensory conditions: visual-tactile matched, visual-tactile mismatched, visual differences only and tactile differences only. A robust MWI was induced across all sensory conditions, whereby the polystyrene object felt heavier than the granite object. The strength of the MWI differed across conditions, with tactile material cues having a stronger influence on perceived heaviness than visual material cues. We discuss how these results suggest a mechanism whereby multisensory integration directly impacts how top-down processes shape perception.

Grasp aperture corrections in reach-to-grasp movements do not reliably alter size perception

When grasping an object, the opening between the fingertips (grip aperture) scales with the size of the object. If an object changes in size, the grip aperture has to be corrected. In this study, it was investigated whether such corrections would influence the perceived size of objects. The grasping plan was manipulated with a preview of the object, after which participants initiated their reaching movement without vision. In a minority of the grasps, the object changed in size after the preview and participants had to adjust their grasping movement. Visual feedback was manipulated in two experiments. In experiment 1, vision was restored during reach and both visual and haptic information was available to correct the grasp and lift the object. In experiment 2, no visual information was provided during the movement and grasps could only be corrected using haptic information. Participants made reach-to-grasp movements towards two objects and compared these in size. Results showed that participants adjusted their grasp to a change in object size from preview to grasped object in both experiments. However, a change in object size did not bias the perception of object size or alter discrimination performance. In experiment 2, a small perceptual bias was found when objects changed from large to small. However, this bias was much smaller than the difference that could be discriminated and could not be considered meaningful. Therefore, it can be concluded that the planning and execution of reach-to-grasp movements do not reliably affect the perception of object size.

A brief glimpse at a haptic target is sufficient for multisensory integration in reaching movements

Goal-directed aiming movements toward visuo-haptic targets (i.e., seen and handheld targets) are generally more precise than those toward visual only or haptic only targets. This multisensory advantage stems from a continuous inflow of haptic and visual target information during the movement planning and execution phases. However, in everyday life, multisensory movements often occur without the support of continuous visual information. Here we investigated whether and to what extent limiting visual information to the initial stage of the action still leads to a multisensory advantage. Participants were asked to reach a handheld target while vision was briefly provided during the movement planning phase (50 ms, 100 ms, 200 ms of vision before movement onset), or during the planning and early execution phases (400 ms of vision), or during the entire movement. Additional conditions were performed in which only haptic target information was provided, or, only vision was provided either briefly (50 ms, 100 ms, 200 ms, 400 ms) or throughout the entire movement. Results showed that 50 ms of vision before movement onset were sufficient to trigger a direction-specific visuo-haptic integration process that increased endpoint precision. We conclude that, when a continuous support of vision is not available, endpoint precision is determined by the less recent, but most reliable multisensory information rather than by the latest unisensory (haptic) inputs.

Reach-to-Grasp: A Multisensory Experience

The reach-to-grasp movement is ordinarily performed in everyday living activities and it represents a key behavior that allows humans to interact with their environment. Remarkably, it serves as an experimental test case for probing the multisensory architecture of goal-oriented actions. This review focuses on experimental evidence that enhances or modifies how we might conceptualize the "multisensory" substrates of prehension. We will review evidence suggesting that how reach-to-grasp movements are planned and executed is influenced by information coming from different sensory modalities such as vision, proprioception, audition, taste, and olfaction. The review closes with some considerations about the predominant role of the multisensory constituents in shaping prehensile behavior and how this might be important for future research developments, especially in the rehabilitative domain.

Integration of haptics and vision in human multisensory grasping

Grasping actions are directed not only toward objects we see but also toward objects we both see and touch (multisensory grasping). In this latter case, the integration of visual and haptic inputs improves movement performance compared to each sense alone. This performance advantage could be due to the integration of all the redundant positional and size cues or to the integration of only a subset of these cues. Here we selectively provided specific cues to tease apart how these different sensory sources contribute to visuo-haptic multisensory grasping. We demonstrate that the availability of the haptic positional cue together with the visual cues is sufficient to achieve the same grasping performance as when all cues are available. These findings provide strong evidence that the human sensorimotor system relies on non-visual sensory inputs and open new perspectives on their role in supporting vision during both development and adulthood.