A comparison of haptic material perception in blind and sighted individuals

Using Immersive Virtual Reality to Examine How Visual and Tactile Cues Drive the Material-Weight Illusion

The material-weight illusion (MWI) demonstrates how our past experience with material and weight can create expectations that influence the perceived heaviness of an object. Here we used mixed-reality to place touch and vision in conflict, to investigate whether the modality through which materials are presented to a lifter could influence the top-down perceptual processes driving the MWI. University students lifted equally-weighted polystyrene, cork and granite cubes whilst viewing computer-generated images of the cubes in virtual reality (VR). This allowed the visual and tactile material cues to be altered, whilst all other object properties were kept constant. Representation of the objects’ material in VR was manipulated to create four sensory conditions: visual-tactile matched, visual-tactile mismatched, visual differences only and tactile differences only. A robust MWI was induced across all sensory conditions, whereby the polystyrene object felt heavier than the granite object. The strength of the MWI differed across conditions, with tactile material cues having a stronger influence on perceived heaviness than visual material cues. We discuss how these results suggest a mechanism whereby multisensory integration directly impacts how top-down processes shape perception.

A Study of Multi-Sensory Experience and Color Recognition in Visual Arts Appreciation of People with Visual Impairment

Visually impaired visitors experience many limitations when visiting museum exhibits, such as a lack of cognitive and sensory access to exhibits or replicas. Contemporary art is evolving in the direction of appreciation beyond simply looking at works, and the development of various sensory technologies has had a great influence on culture and art. Thus, opportunities for people with visual impairments to appreciate visual artworks through various senses such as hearing, touch, and smell are expanding. However, it is uncommon to provide an interactive interface for color recognition, such as applying patterns, sounds, temperature, or scents. This review aims to convey the visual elements of the work to the visually impaired through various sensory elements. In addition, to open a new perspective on appreciation of the works, the technique of expressing the color coded by integrating patterns, temperature, scent, music, and vibration was explored, and future research topics were presented.

Virtual Haptic Perception as an Educational Assistive Technology: A Case Study in Inclusive Education

The traditional educational process of blind people is a complex practice that relies on the haptic perception (tactile perception) of physical models. However, physical models may be costly, inaccessible or may require a large storage space. To overcome these difficulties, in this article a virtual haptic perception approach to support the teaching and learning process of blind people is proposed. The proposed approach combines the use of virtual reality and haptic technologies. The research aim is to objectively evaluate the feasibility and effectiveness of using virtual haptic perception in the education of blind children. For this purpose, an experimental methodology was defined and used to teach maths, in particular fundamental three-dimensional (3D) shapes, to blind children. The results are analysed in terms of the participants' ability to explore and recognize virtual objects, and the knowledge gain after the virtual perception learning period. From this analysis it is concluded that haptic virtual perception is a valid and effective assistive technology for the education of blind children.

Visuo-haptic transfer for object recognition in children with peripheral visual impairment

It is well known how early visual experience is critical for the development of multisensory processing abilities, and for this reason an early vision impairment could hinder the transfer of different sensory information during the exploration and recognition of the surrounding environment. Recently, we verified that visuo-haptic transfer for object recognition emerges early in typically developing children but matures slowly during the school-age period. Subsequently we verified the presence of a slower trend of development in unisensory and multisensory skills in children with early abnormal motor and sensory experiences due to brain lesions. Now, we investigated unimodal visual information, unimodal haptic information and visuo-haptic information transfer in children with a diagnosis of low-vision, due to congenital visual impairment. Unimodal and bimodal processes for object recognition were explored in 11 children with low-vision and the results were matched with those of 22 controls. Participants were tested using a clinical protocol involving visual exploration of black-and-white photographs of common objects, haptic exploration of real objects and visuo-haptic transfer of these two types of information. Results show a normal development in haptic unisensory processing in children with low vision and a significant difference in multisensory transfer between the two groups. In children with visual impairment, multisensory processes do not facilitate the recognition of common objects as in typical children, probably because early visual impairment may impact the cross-sensory calibration of vision and touch.

Mental Imagery Follows Similar Cortical Reorganization as Perception: Intra-Modal and Cross-Modal Plasticity in Congenitally Blind

Cortical plasticity in congenitally blind individuals leads to cross-modal activation of the visual cortex and may lead to superior perceptual processing in the intact sensory domains. Although mental imagery is often defined as a quasi-perceptual experience, it is unknown whether it follows similar cortical reorganization as perception in blind individuals. In this study, we show that auditory versus tactile perception evokes similar intra-modal discriminative patterns in congenitally blind compared with sighted participants. These results indicate that cortical plasticity following visual deprivation does not influence broad intra-modal organization of auditory and tactile perception as measured by our task. Furthermore, not only the blind, but also the sighted participants showed cross-modal discriminative patterns for perception modality in the visual cortex. During mental imagery, both groups showed similar decoding accuracies for imagery modality in the intra-modal primary sensory cortices. However, no cross-modal discriminative information for imagery modality was found in early visual cortex of blind participants, in contrast to the sighted participants. We did find evidence of cross-modal activation of higher visual areas in blind participants, including the representation of specific-imagined auditory features in visual area V4.

Aging and the Haptic Perception of Material Properties

The ability of 26 younger (mean age was 22.5 years) and older adults (mean age was 72.6 years) to haptically perceive material properties was evaluated. The participants manually explored (for 5 seconds) 42 surfaces twice and placed each of these 84 experimental stimuli into one of seven categories: paper, plastic, metal, wood, stone, fabric, and fur/leather. In general, the participants were best able to identify fur/leather and wood materials; in contrast, recognition performance was worst for stone and paper. Despite similar overall patterns of performance for younger and older participants, the younger adults’ recognition accuracies were 26.5% higher. The participants’ tactile acuities (assessed by tactile grating orientation discrimination) affected their ability to identify surface material. In particular, the Pearson r correlation coefficient relating the participants’ grating orientation thresholds and their material identification performance was −0.8: The higher the participants’ thresholds, the lower the material recognition ability. While older adults are able to effectively perceive the solid shape of environmental objects using the sense of touch, their ability to perceive surface materials is significantly compromised.

Image Statistics and the Representation of Material Properties in the Visual Cortex

We explored perceived material properties (roughness, texturedness, and hardness) with a novel approach that compares perception, image statistics and brain activation, as measured with fMRI. We initially asked participants to rate 84 material images with respect to the above mentioned properties, and then scanned 15 of the participants with fMRI while they viewed the material images. The images were analyzed with a set of image statistics capturing their spatial frequency and texture properties. Linear classifiers were then applied to the image statistics as well as the voxel patterns of visually responsive voxels and early visual areas to discriminate between images with high and low perceptual ratings. Roughness and texturedness could be classified above chance level based on image statistics. Roughness and texturedness could also be classified based on the brain activation patterns in visual cortex, whereas hardness could not. Importantly, the agreement in classification based on image statistics and brain activation was also above chance level. Our results show that information about visual material properties is to a large degree contained in low-level image statistics, and that these image statistics are also partially reflected in brain activity patterns induced by the perception of material images.

Statistical correlates of perceived gloss in natural images

It is currently debated whether the perception of gloss is linked to the statistical parameters of the retinal image. In particular, it has been suggested that gloss is highly correlated with the skewness of the luminance histogram. However, other psychophysical work with artificial stimuli has shown that skewness alone is not enough to induce the perception of gloss. Here, we analyzed many images of natural surfaces to search for potential statistical correlates of perceived gloss. We found that skewness indeed correlates with gloss when using rendered stimuli, but that the standard deviation, a measure of contrast, correlates better with perceived gloss when using photographs of natural surfaces. We verified the important role of contrast by manipulating skewness and contrast within images. Changing the contrast in images significantly modulates perceived gloss, but manipulating the skewness of the luminance histogram had only a small effect.