Visual mediation and the haptic recognition of two-dimensional pictures of common objects

Recognition of facial expressions of emotions in tactile drawings by blind children, children with low vision and sighted children

In the context of blindness, studies on the recognition of facial expressions of emotions by touch are essential to define the compensatory touch abilities and to create adapted tools on emotions. This study is the first to examine the effect of visual experience in the recognition of tactile drawings of facial expressions of emotions by children with different visual experiences. To this end, we compared the recognition rates of tactile drawings of emotions between blind children, children with low vision and sighted children aged 6-12 years. Results revealed no effect of visual experience on recognition rates. However, an effect of emotions and an interaction effect between emotions and visual experience were found. Indeed, while all children had a low average recognition rate, the drawings of fear, anger and disgust were particularly poorly recognized. Moreover, sighted children were significantly better at recognizing the drawings of surprise and sadness than the blind children who only showed high recognition rates for joy. The results of this study support the importance of developing emotion tools that can be understood by children with different visual experiences.

Prior visual experience increases children's use of effective haptic exploration strategies in audio-tactile sound-shape correspondences

Sound-shape correspondence refers to the preferential mapping of information across the senses, such as associating a nonsense word like bouba with rounded abstract shapes and kiki with spiky abstract shapes. Here we focused on audio-tactile (AT) sound-shape correspondences between nonsense words and abstract shapes that are felt but not seen. Despite previous research indicating a role for visual experience in establishing AT associations, it remains unclear how visual experience facilitates AT correspondences. Here we investigated one hypothesis: seeing the abstract shapes improve haptic exploration by (a) increasing effective haptic strategies and/or (b) decreasing ineffective haptic strategies. We analyzed five haptic strategies in video-recordings of 6- to 8-year-old children obtained in a previous study. We found the dominant strategy used to explore shapes differed based on visual experience. Effective strategies, which provide information about shape, were dominant in participants with prior visual experience, whereas ineffective strategies, which do not provide information about shape, were dominant in participants without prior visual experience. With prior visual experience, poking-an effective and efficient strategy-was dominant, whereas without prior visual experience, uncategorizable and ineffective strategies were dominant. These findings suggest that prior visual experience of abstract shapes in 6- to 8-year-olds can increase the effectiveness and efficiency of haptic exploration, potentially explaining why prior visual experience can increase the strength of AT sound-shape correspondences.

Drawing as a tool for investigating the nature of imagery representations of blind people: The case of the canonical size phenomenon

Several studies have shown that blind people, including those with congenital blindness, can use raised-line drawings, both for "reading" tactile graphics and for drawing unassisted. However, research on drawings produced by blind people has mainly been qualitative. The current experimental study was designed to investigate the under-researched issue of the size of drawings created by people with blindness. Participants (N = 59) varied in their visual status. Adventitiously blind people had previous visual experience and might use visual representations (e.g., when visualising objects in imagery/working memory). Congenitally blind people did not have any visual experience. The participant's task was to draw from memory common objects that vary in size in the real world. The findings revealed that both groups of participants produced larger drawings of objects that have larger actual sizes. This means that the size of familiar objects is a property of blind people's mental representations, regardless of their visual status. Our research also sheds light on the nature of the phenomenon of canonical size. Since we have found the canonical size effect in a group of people who are blind from birth, the assumption of the visual nature of this phenomenon - caused by the ocular-centric biases present in studies on drawing performance - should be revised.

Similar object shape representation encoded in the inferolateral occipitotemporal cortex of sighted and early blind people

We can sense an object's shape by vision or touch. Previous studies suggested that the inferolateral occipitotemporal cortex (ILOTC) implements supramodal shape representations as it responds more to seeing or touching objects than shapeless textures. However, such activation in the anterior portion of the ventral visual pathway could be due to the conceptual representation of an object or visual imagery triggered by touching an object. We addressed these possibilities by directly comparing shape and conceptual representations of objects in early blind (who lack visual experience/imagery) and sighted participants. We found that bilateral ILOTC in both groups showed stronger activation during a shape verification task than during a conceptual verification task made on the names of the same manmade objects. Moreover, the distributed activity in the ILOTC encoded shape similarity but not conceptual association among objects. Besides the ILOTC, we also found shape representation in both groups' bilateral ventral premotor cortices and intraparietal sulcus (IPS), a frontoparietal circuit relating to object grasping and haptic processing. In contrast, the conceptual verification task activated both groups' left perisylvian brain network relating to language processing and, interestingly, the cuneus in early blind participants only. The ILOTC had stronger functional connectivity to the frontoparietal circuit than to the left perisylvian network, forming a modular structure specialized in shape representation. Our results conclusively support that the ILOTC selectively implements shape representation independently of visual experience, and this unique functionality likely comes from its privileged connection to the frontoparietal haptic circuit.

Cognitive map formation in the blind is enhanced by three-dimensional tactile information

For blind individuals, tactile maps are useful tools to form cognitive maps through touch. However, they still experience challenges in cognitive map formation and independent navigation. Three-dimensional (3D) tactile information is thus increasingly being considered to convey enriched spatial information, but it remains unclear if it can facilitate cognitive map formation compared to traditional two-dimensional (2D) tactile information. Consequently, the present study investigated the impact of the type of sensory input (tactile 2D vs. tactile 3D vs. a visual control condition) on cognitive map formation. To do so, early blind (EB, n = 13), late blind (LB, n = 12), and sighted control (SC, n = 14) participants were tasked to learn the layouts of mazes produced with different sensory information (tactile 2D vs. tactile 3D vs. visual control) and to infer routes from memory. Results show that EB manifested stronger cognitive map formation with 3D mazes, LB performed equally well with 2D and 3D tactile mazes, and SC manifested equivalent cognitive map formation with visual and 3D tactile mazes but were negatively impacted by 2D tactile mazes. 3D tactile maps therefore have the potential to improve spatial learning for EB and newly blind individuals through a reduction of cognitive overload. Installation of 3D tactile maps in public spaces should be considered to promote universal accessibility and reduce blind individuals' wayfinding deficits related to the inaccessibility of spatial information through non-visual means.

Functional relevance of the extrastriate body area for visual and haptic object recognition: a preregistered fMRI-guided TMS study

The extrastriate body area (EBA) is a region in the lateral occipito-temporal cortex (LOTC), which is sensitive to perceived body parts. Neuroimaging studies suggested that EBA is related to body and tool processing, regardless of the sensory modalities. However, how essential this region is for visual tool processing and nonvisual object processing remains a matter of controversy. In this preregistered fMRI-guided repetitive transcranial magnetic stimulation (rTMS) study, we examined the causal involvement of EBA in multisensory body and tool recognition. Participants used either vision or haptics to identify 3 object categories: hands, teapots (tools), and cars (control objects). Continuous theta-burst stimulation (cTBS) was applied over left EBA, right EBA, or vertex (control site). Performance for visually perceived hands and teapots (relative to cars) was more strongly disrupted by cTBS over left EBA than over the vertex, whereas no such object-specific effect was observed in haptics. The simulation of the induced electric fields confirmed that the cTBS affected regions including EBA. These results indicate that the LOTC is functionally relevant for visual hand and tool processing, whereas the rTMS over EBA may differently affect object recognition between the 2 sensory modalities.

Early blindness modulates haptic object recognition

Haptic object recognition is usually an efficient process although slower and less accurate than its visual counterpart. The early loss of vision imposes a greater reliance on haptic perception for recognition compared to the sighted. Therefore, we may expect that congenitally blind persons could recognize objects through touch more quickly and accurately than late blind or sighted people. However, the literature provided mixed results. Furthermore, most of the studies on haptic object recognition focused on performance, devoting little attention to the exploration procedures that conducted to that performance. In this study, we used iCube, an instrumented cube recording its orientation in space as well as the location of the points of contact on its faces. Three groups of congenitally blind, late blind and age and gender-matched blindfolded sighted participants were asked to explore the cube faces where little pins were positioned in varying number. Participants were required to explore the cube twice, reporting whether the cube was the same or it differed in pins disposition. Results showed that recognition accuracy was not modulated by the level of visual ability. However, congenitally blind touched more cells simultaneously while exploring the faces and changed more the pattern of touched cells from one recording sample to the next than late blind and sighted. Furthermore, the number of simultaneously touched cells negatively correlated with exploration duration. These findings indicate that early blindness shapes haptic exploration of objects that can be held in hands.

Size and Quality of Drawings Made by Adults Under Visual and Haptic Control

The aim of this study was twofold. First, our objective was to test the influence of an object’s actual size (size rank) on the drawn size of the depicted object. We tested the canonical size effect (i.e., drawing objects larger in the physical world as larger) in four drawing conditions — two perceptual conditions (blindfolded or sighted) crossed with two materials (paper or special foil for producing embossed drawings). Second, we investigated whether drawing quality (we analysed both the local and global criteria of quality) depends on drawing conditions. We predicted that drawing quality, unlike drawing size, would vary according to drawing conditions — namely, being higher when foil than paper was used for drawing production in the blindfolded condition. We tested these hypotheses with young adults who repeatedly drew eight different familiar objects (differentiated by size in the real world) in four drawing conditions. As expected, drawn size increased linearly with increasing size rank, whatever the drawing condition, thus replicating the canonical size effect and showing that this effect was not dependent on drawing conditions. In line with our hypothesis, in the blindfolded condition drawing quality was better when foil rather than paper was used, suggesting a benefit from haptic feedback on the trace produced. Besides, the quality of drawings produced was still higher in the sighted than the blindfolded condition. In conclusion, canonical size is present under different drawing conditions regardless of whether sight is involved or not, while perceptual control increases drawing quality in adults.

Communicating Photograph Content Through Tactile Images to People With Visual Impairments

Millions of people with a visual impairment across the world are denied access to visual images. They are unable to enjoy the simple pleasures of viewing family photographs, those in textbooks or tourist brochures and the pictorial embellishment of news stories etc. We propose a simple, inexpensive but effective approach, to make content accessible via touch. We use state-of-the-art algorithms to automatically process an input photograph into a collage of icons, that depict the most important semantic aspects of a scene. This collage is then printed onto swell paper. Our experiments show that people can recognise content with an accuracy exceeding 70% and create plausible narratives to explain it. This means that people can understand image content via touch. Communicating scene foreground is a step forward, but there are many other steps needed to provide the visually impaired with the fullest possible access to visual content.

Haptic recognition memory and lateralisation for verbal and nonverbal shapes

Laterality effects generally refer to an advantage for verbal processing in the left hemisphere and for non-verbal processing in the right hemisphere, and are often demonstrated in memory tasks in vision and audition. In contrast, their role in haptic memory is less understood. In this study, we examined haptic recognition memory and laterality for letters and nonsense shapes. We used both upper and lower case letters, with the latter designed as more complex in shape. Participants performed a recognition memory task with the left and right hand separately. Recognition memory performance (capacity and bias-free d') was higher and response times were faster for upper case letters than for lower case letters and nonsense shapes. The right hand performed best for upper case letters when it performed the task after the left hand. This right hand/left hemisphere advantage appeared for upper case letters, but not lower case letters, which also had a lower memory capacity, probably due to their more complex spatial shape. These findings suggest that verbal laterality effects in haptic memory are not very prominent, which may be due to the haptic verbal stimuli being processed mainly as spatial objects without reaching robust verbal coding into memory.

The roles of lower- and higher-order surface statistics in tactile texture perception

Humans can haptically discriminate surface textures when there is a significant difference in the statistics of the surface profile. Previous studies on tactile texture discrimination have emphasized the perceptual effects of lower-order statistical features such as carving depth, inter-ridge distance, and anisotropy, which can be characterized by local amplitude spectra or spatial-frequency/orientation subband histograms. However, the real-world surfaces we encounter in everyday life also differ in the higher-order statistics, such as statistics about correlations of nearby spatial-frequencies/orientations. For another modality, vision, the human brain has the ability to use the textural differences in both higher- and lower-order image statistics. In this work, we examined whether the haptic texture perception can use higher-order surface statistics as visual texture perception does, by three-dimensional (3-D)-printing textured surfaces transcribed from different "photos" of natural scenes such as stones and leaves. Even though the maximum carving depth was well above the haptic detection threshold, some texture pairs were hard to discriminate. Specifically, those texture pairs with similar amplitude spectra were difficult to discriminate, which suggests that the lower-order statistics have the dominant effect on tactile texture discrimination. To directly test the poor sensitivity of the tactile texture perception to higher-order surface statistics, we matched the lower-order statistics across different textures using a texture synthesis algorithm and found that haptic discrimination of the matched textures was nearly impossible unless the stimuli contained salient local features. We found no evidence for the ability of the human tactile system to use higher-order surface statistics for texture discrimination.NEW & NOTEWORTHY Humans can discriminate subtle spatial patterns differences in the surrounding world through their hands, but the underlying computation remains poorly understood. Here, we 3-D-printed textured surfaces and analyzed the tactile discrimination performance regarding the sensitivity to surface statistics. The results suggest that observers have sensitivity to lower-order statistics whereas not to higher-order statistics. That is, touch differs from vision not only in spatiotemporal resolution but also in (in)sensitivity to high-level surface statistics.

The feeling of "kiki": Comparing developmental changes in sound-shape correspondence for audio-visual and audio-tactile stimuli

Sound-shape crossmodal correspondence, the naturally occurring associations between abstract visual shapes and nonsense sounds, is one aspect of multisensory processing that strengthens across early childhood. Little is known regarding whether school-aged children exhibit other variants of sound-shape correspondences such as audio-tactile (AT) associations between tactile shapes and nonsense sounds. Based on previous research in blind individuals suggesting the role of visual experience in establishing sound-shape correspondence, we hypothesized that children would show weaker AT association than adults and that children's AT association would be enhanced with visual experience of the shapes. In Experiment 1, we showed that, when asked to match shapes explored haptically via touch to nonsense words, 6- to 8-year-olds exhibited inconsistent AT associations, whereas older children and adults exhibited the expected AT associations, despite robust audio-visual (AV) associations found across all age groups in a related study. In Experiment 2, we confirmed the role of visual experience in enhancing AT association; here, 6- to 8-year-olds could exhibit the expected AT association if first exposed to the AV condition, whereas adults showed the expected AT association irrespective of whether the AV condition was tested first or second. Our finding suggests that AT sound-shape correspondence is weak early in development relative to AV sound-shape correspondence, paralleling previous findings on the development of other types of multisensory associations. The potential role of visual experience in the development of sound-shape correspondences in other senses is discussed.

Exploring 3D miniatures with action simulations by finger gestures: Study of a new embodied design for blind and sighted children

Tactile books for blind children generally contain tactile illustrations referring to a visual world that can be difficult to understand. This study investigates an innovative way to present content to be explored by touch. Following embodied approaches and evidence about the advantages of manipulations in tactile processing, we examined 3D miniatures that children explored using their middle and index fingers to simulate leg movements. This “Action simulations by finger gestures–ASFG” procedure has a symbolic relevance in the context of blindness. The aim of the present study was to show how the ASFG procedure facilitates the identification of objects by blind and sighted children. Experiment 1 examined the identification of 3D miniatures of action objects (e.g. the toboggan, trampoline) by 8 early blind and 15 sighted children, aged 7 to 12, who explored with the ASFG procedure. Results revealed that objects were very well identified by the two groups of children. Results confirmed hypotheses that ASFG procedures are relevant in the identification process regardless of the visual status of subjects. Experiment (control) 2 studied identification of tactile pictures of same action objects by 8 different early blind and 15 sighted children, aged 7 to 12. Results confirmed that almost all objects obtained lower recognition scores in tactile pictures than in 3D miniatures by both groups and showed surprisingly higher scores in blind children than in sighted children. Taken together, our study provides evidence of the contribution of sensorimotor simulation in the identification of objects by touch and brings innovative solutions in book design for blind people. Moreover, it means that only the ASFG procedure has a very inclusive potential to be relevant for a larger number of subjects, regardless of their visual skills.

The Nature of Haptic Working Memory Capacity and Its Relation to Visual Working Memory

I conducted three experiments to investigate haptic working memory capacity using a haptic change detection task with 2D stimuli. I adopted a single-task paradigm comprising haptic single-feature (orientation or texture) and haptic multifeature (orientation and texture) conditions in Experiment 1 and a dual-task paradigm with a primary haptic orientation or texture change detection task and a concurrent secondary visual shape or colour change detection task in Experiments 2-3. I observed that in the single-task paradigm, haptic change detection capacity was higher for single features than it was for multiple features. In haptic working memory, unlike in visual working memory, features of two different dimensions within an object cannot be integrated. In the dual-task paradigm, interference was observed when the concurrent visual shape change detection task was combined with the haptic orientation change detection task although interference was not observed when the concurrent visual colour change detection task was combined with it. In addition, the concurrent visual shape or colour change detection task did not interfere with the capacity for haptic texture memory, which was higher than that for haptic orientation memory. These findings demonstrate that geometric properties perhaps retained a common storage system shared between haptic working memory and visual working memory; however, haptic texture might be retained in an independent stable storage system that is haptic-specific.

A Code for Reducing Figure-Ground Ambiguities in Tactile Graphics

This article describes a code that could facilitate the interpretation of tactile outline graphics by reducing potential ambiguities in figure-ground, slope, and curvature. An experiment tested the ability of six blind adults to learn the code quickly, locate and identify all symbols tactilely, and use the code to match embossed drawings to complex models that were designed to present line-referent ambiguities. All the subjects learned the code in less than two hours and successfully matched coded graphics to targeted objects.

Crossmodal learning of target-context associations: When would tactile context predict visual search?

It is well established that statistical learning of visual target locations in relation to constantly positioned visual distractors facilitates visual search. In the present study, we investigated whether such a contextual-cueing effect would also work crossmodally, from touch onto vision. Participants responded to the orientation of a visual target singleton presented among seven homogenous visual distractors. Four tactile stimuli, two to different fingers of each hand, were presented either simultaneously with or prior to the visual stimuli. The identity of the stimulated fingers provided the crossmodal context cue: in half of the trials, a given visual target location was consistently paired with a given tactile configuration. The visual stimuli were presented above the unseen fingers, ensuring spatial correspondence between vision and touch. We found no evidence of crossmodal contextual cueing when the two sets of items (tactile, visual) were presented simultaneously (Experiment 1). However, a reliable crossmodal effect emerged when the tactile distractors preceded the onset of visual stimuli 700 ms (Experiment 2). But crossmodal cueing disappeared again when, after an initial learning phase, participants flipped their hands, making the tactile distractors appear at different positions in external space while their somatotopic positions remained unchanged (Experiment 3). In all experiments, participants were unable to explicitly discriminate learned from novel multisensory arrays. These findings indicate that search-facilitating context memory can be established across vision and touch. However, in order to guide visual search, the (predictive) tactile configurations must be remapped from their initial somatotopic into a common external representational format.

Identification of Textured Tactile Pictures in Visually Impaired and Blindfolded Sighted Children

A high level of variability in the capacity of visually impaired children to accurately identify tactile images is reported in the literature, with on average rather low percentages of correct naming responses. However, most of these studies used raised-line drawings as stimuli to be explored and named. The present experiment investigated whether blind children of 3 to 8 years of age would demonstrate a satisfactory ability to name the elements making up tactile images when tested in an experimental setting similar to their natural reading conditions. Textured tactile images taken from genuine illustrated tactile books for young children were used, and the participants received information about the title of the book or listened the text that accompanied each picture before exploration, as it would occur in a natural reading setting. The results showed that their naming scores were indeed higher than previously reported at equivalent ages and did not differ from those of age-matched sighted children. These scores were positively impacted by haptic practice in blind children and correlated with the use of some specific exploratory procedures. The blind children benefited from information provided before exploration, as did their sighted counterparts. However, only in the former did the condition in which full information was provided influence the way the children organized their exploration. The haptic identification scores increased with age regardless of visual status, with the exploration times decreasing in the blind children, while the reverse trend was observed in the sighted children. These results are discussed at the light of the image-mediation model of haptics, suggesting that during the age period considered in the present experiment, blind children would progressively learn to process haptic information directly, thus leading to a decrease of exploration times, while sighted children would learn to translate haptic information into a visual image used to retrieve semantic information, involving an increase of their exploration times.

Performance of Blind and Sighted Persons on Spatial Tasks

This article reports on a study of the performance by congenitally blind, adventitiously blind, and sighted persons on three types of tasks: manipulatory, simple locomotion, and complex locomotion. The three groups of subjects tended to perform equivalently, and the results offer little evidence of a set of spatial processes that rely on past visual experience and are applicable to a broad variety of tasks.

Reinterpretation in visual imagery is possible without visual cues: a validation of previous research

Is visual reinterpretation of bistable figures (e.g., duck/rabbit figure) in visual imagery possible? Current consensus suggests that it is in principle possible because of converging evidence of quasi-pictorial functioning of visual imagery. Yet, studies that have directly tested and found evidence for reinterpretation in visual imagery, allow for the possibility that reinterpretation was already achieved during memorization of the figure(s). One study resolved this issue, providing evidence for reinterpretation in visual imagery (Mast and Kosslyn, Cognition 86:57-70, 2002). However, participants in that study performed reinterpretations with aid of visual cues. Hence, reinterpretation was not performed with mental imagery alone. Therefore, in this study we assessed the possibility of reinterpretation without visual support. We further explored the possible role of haptic cues to assess the multimodal nature of mental imagery. Fifty-three participants were consecutively presented three to be remembered bistable 2-D figures (reinterpretable when rotated 180°), two of which were visually inspected and one was explored hapticly. After memorization of the figures, a visually bistable exemplar figure was presented to ensure understanding of the concept of visual bistability. During recall, 11 participants (out of 36; 30.6%) who did not spot bistability during memorization successfully performed reinterpretations when instructed to mentally rotate their visual image, but additional haptic cues during mental imagery did not inflate reinterpretation ability. This study validates previous findings that reinterpretation in visual imagery is possible.

Enhancing general spatial skills of young visually impaired people with a programmable distance discrimination training: a case control study

BACKGROUND

The estimation of relative distance is a perceptual task used extensively in everyday life. This important skill suffers from biases that may be more pronounced when estimation is based on haptics. This is especially true for the blind and visually impaired, for which haptic estimation of distances is paramount but not systematically trained. We investigated whether a programmable tactile display, used autonomously, can improve distance discrimination ability in blind and severely visually impaired youngsters between 7 and 22 years-old.

METHODS

Training consisted of four weekly sessions in which participants were asked to haptically find, on the programmable tactile display, the pairs of squares which were separated by the shortest and longest distance in tactile images with multiple squares. A battery of haptic tests with raised-line drawings was administered before and after training, and scores were compared to those of a control group that did only the haptic battery, without doing the distance discrimination training on the tactile display.

RESULTS

Both blind and severely impaired youngsters became more accurate and faster at the task during training. In haptic battery results, blind and severely impaired youngsters who used the programmable display improved in three and two tests, respectively. In contrast, in the control groups, the blind control group improved in only one test, and the severely visually impaired in no tests.

CONCLUSIONS

Distance discrimination skills can be trained equally well in both blind and severely impaired participants. More importantly, autonomous training with the programmable tactile display had generalized effects beyond the trained task. Participants improved not only in the size discrimination test but also in memory span tests. Our study shows that tactile stimulation training that requires minimal human assistance can effectively improve generic spatial skills.

Brain networks underlying tactile softness perception: A functional magnetic resonance imaging study

Humans are adept at perceiving physical properties of an object through touch. Tangible object properties can be categorized into two types: macro-spatial properties, including shape and orientation; and material properties, such as roughness, softness, and temperature. Previous neuroimaging studies have shown that roughness and temperature are extracted at nodes of a network, such as that involving the parietal operculum and insula, which is different from the network engaged in processing macro-spatial properties. However, it is unclear whether other perceptual dimensions pertaining to material properties engage the same regions. Here, we conducted a functional magnetic resonance imaging study to test whether the parietal operculum and insula were involved in extracting tactually-perceived softness magnitude. Fifty-six healthy right-handed participants estimated perceived softness magnitude using their right middle finger. We presented three stimuli that had the same shape but different compliances. The force applied to the finger was manipulated at two levels. Classical mass-univariate analysis showed that activity in the parietal operculum, insula, and medial prefrontal cortex was positively associated with perceived softness magnitude, regardless of the applied force. Softness-related activity was stronger in the ventral striatum in the high-force condition than in the low-force condition. The multivariate voxel pattern analysis showed higher accuracy than chance levels and control regions in the parietal operculum/insula, postcentral gyrus, posterior parietal lobule, and middle occipital gyrus. These results indicate that a distributed set of the brain regions, including the parietal operculum and insula, is involved in representing perceived softness.

Adults' visual recognition of actions simulations by finger gestures (ASFGs) produced by sighted and blind individuals

The present study examines the visual recognition of action simulations by finger gestures (ASFGs) produced by sighted and blind individuals. In ASFGs, fingers simulate legs to represent actions such as jumping, spinning, climbing, etc. The question is to determine whether the common motor experience of one’s own body is sufficient to produce adequate ASFGs or whether the possibility to see gestures from others are also necessary to do it. Three experiments were carried out to address this question. Experiment 1 examined in 74 sighted adults the recognition of 18 types of ASFGs produced by 20 blindfolded sighted adults. Results showed that rates of correct recognition were globally very high, but varied with the type of ASFG. Experiment 2 studied in 91 other sighted adults the recognition of ASFGs produced by 10 early blind and 7 late blind adults. Results also showed a high level of recognition with a similar order of recognizability by type of ASFG. However, ASFGs produced by early blind individuals were more poorly recognized than those produced by late blind individuals. In order to match data of recognition obtained with the form that gestures are produced by individuals, two independant judges evaluated prototypical and atypical attributes of ASFG produced by blindfolded sighted, early blind and late blind individuals in Experiment 3. Results revealed the occurrence of more atypical attributes in ASFG produced by blind individuals: their ASFGs transpose more body movements from a character-viewpoint in less agreement with visual rules. The practical interest of the study relates to the relevance of including ASFGs as a new exploratory procedure in tactile devices which are more apt to convey action concepts to blind users/readers.

The role of visual experience in the production of emotional facial expressions by blind people: a review

Facial expressions of emotion are nonverbal behaviors that allow us to interact efficiently in social life and respond to events affecting our welfare. This article reviews 21 studies, published between 1932 and 2015, examining the production of facial expressions of emotion by blind people. It particularly discusses the impact of visual experience on the development of this behavior from birth to adulthood. After a discussion of three methodological considerations, the review of studies reveals that blind subjects demonstrate differing capacities for producing spontaneous expressions and voluntarily posed expressions. Seventeen studies provided evidence that blind and sighted spontaneously produce the same pattern of facial expressions, even if some variations can be found, reflecting facial and body movements specific to blindness or differences in intensity and control of emotions in some specific contexts. This suggests that lack of visual experience seems to not have a major impact when this behavior is generated spontaneously in real emotional contexts. In contrast, eight studies examining voluntary expressions indicate that blind individuals have difficulty posing emotional expressions. The opportunity for prior visual observation seems to affect performance in this case. Finally, we discuss three new directions for research to provide additional and strong evidence for the debate regarding the innate or the culture-constant learning character of the production of emotional facial expressions by blind individuals: the link between perception and production of facial expressions, the impact of display rules in the absence of vision, and the role of other channels in expression of emotions in the context of blindness.

Representations of microgeometric tactile information during object recognition

Object recognition through tactile perception involves two elements: the shape of the object (macrogeometric properties) and the material of the object (microgeometric properties). Here we sought to determine the characteristics of microgeometric tactile representations regarding object recognition through tactile perception. Participants were directed to recognize objects with different surface materials using either tactile information or visual information. With a quantitative analysis of the cognitive process regarding object recognition, Experiment 1 confirmed the same eight concepts (composed of rules defining distinct cognitive processes) commonly generated in both tactile and visual perceptions to accomplish the task, although an additional concept was generated during the visual task. Experiment 2 focused only on tactile perception. Three tactile objects with different surface materials (plastic, cloth and sandpaper) were used for the object recognition task. The participants answered a questionnaire regarding the process leading to their answers (which was designed based on the results obtained in Experiment 1) and to provide ratings on the vividness, familiarity and affective valence. We used these experimental data to investigate whether changes in material attributes (tactile information) change the characteristics of tactile representation. The observation showed that differences in tactile information resulted in differences in cognitive processes, vividness, familiarity and emotionality. These two experiments collectively indicated that microgeometric tactile information contributes to object recognition by recruiting various cognitive processes including episodic memory and emotion, similar to the case of object recognition by visual information.

Visual, tangible, and touch-screen: Comparison of platforms for displaying simple graphics

Four different platforms were compared in a task of exploring an angular stimulus and reporting its value. The angle was explored visually, tangibly as raised fine-grit sandpaper, or on a touch-screen with a frictional or vibratory signal. All platforms produced highly accurate angle judgments. Differences were found, however, in exploration time, with vision fastest as expected, followed by tangible, vibration, and friction. Relative to the tangible display, touch-screens evidenced greater noise in the perceived angular value, with a particular disadvantage for friction. The latter must be interpreted in the context of a first-generation display and a rapidly advancing technology. On the whole, the results point both to promise and barriers in the use of refreshable graphical displays for blind users.

Haptics and Projection: Drawings by Tracy, a Blind Adult

Outline drawings in a raised form were made by a blind woman, Tracy, who has been blind from very early in life. Highly practiced in drawing, she reports she is largely self-taught. To invoke matters of projection, she was asked to represent an object with faces slanting away from the observer, a fixed array from different vantage points, and sets of objects in depth. In particular, she drew a cube balanced on a vertex, three objects from different vantage points, receding rows of glasses, and a house. Her drawings included features of parallel and polar projection. Her use of these features may reflect an appreciation of direction from a vantage point, which observers deal with via haptics in everyday tasks. Tracy may have advanced drawing-development skills common to the blind and the sighted.

Beyond visual conventions: Rethinking the design of tactile diagrams

This article reviews the debate over the comprehension of two-dimensional microcapsule tactile pictures portraying three-dimensional information through the use of perspective and visual drawing conventions. It is proposed that three-dimensional haptic object recognition and its relationship to visual object recognition can inform the design of two-dimensional tactual displays. Experimental work is discussed which validates the use of such information in informing the design of 2-D tactile pictures.

A Haptic Face-Inversion Effect

We examined whether a face-inversion effect occurs when participants explore faces by touch. We used a haptic version of the inversion paradigm with 3-D clay facemasks and non-face control objects (teapots) moulded from real objects. Young, neurologically intact, blindfolded participants performed a temporally unconstrained haptic same/different task in each of four stimulus conditions: upright facemasks, inverted facemasks, upright teapots, and inverted teapots. There was a significant inversion effect for faces in terms of accuracy, but none for teapots. The results are considered in terms of the consequences of sequential manual exploration for haptic face processing.

Angle Discrimination in Raised-Line Drawings

We investigated the angular resolution subserving the haptic perception of raised-line drawings by measuring how accurately observers could discriminate between two angle sizes under various conditions. We found that, for acute angles, discrimination performance is highly dependent on exploration strategy: mean thresholds of 2.9° and 6.0° were found for two different exploration strategies. For one of the strategies we found that discriminability is not dependent on the bisector orientation of the angle. Furthermore, we found that thresholds almost double when the angular extent is increased from 20° to 135°. We also found that local apex information has a significant influence on discrimination for acute as well as obtuse angles. In the last experiment we investigated the influence of depiction mode but did not find any effect. Overall, the results tell us that the acuity with which angles in raised-line drawings are perceived is determined by the exploration strategy, local apex information, and global angular extent.

Haptic Pictures: Fit Judgments Predict Identification, Recognition Memory, and Confidence

D'Angiulli et al (1998 Scandinavian Journal of Psychology39 187–190) found blind and sighted (blindfolded) children identified common objects in raised-outline drawings explored haptically, and corrected themselves without feedback. The self-correction suggests that participants can assess the extent to which the referents they suggest as possible identifications fit the haptic pictures. Indeed, when we asked subjects to identify haptic pictures, and to judge how well the referents they mentioned fitted the pictures, their fit judgments predicted the accuracy of their suggestions. Also, when one group of subjects offered the suggestions and another group assessed the fit of the suggestions to the pictures, the fit judgments predicted the accuracy of the suggestions. Further, good fit predicted successful recognition memory. In addition, both high and low fit judgments were made confidently, so the range of confidence judgments was smaller than the range of fit judgments. Finally, visual judgments of fit by one group predicted the level of success of the suggestions from another (haptic) group. In sum, subjects assess their suggested identifications appropriately, most likely on the basis of object shape criteria, outlined surface edges, and use of a vantage point.

Viewpoint and Orientation Influence Picture Recognition in the Blind

In the first three experiments, subjects felt solid geometrical forms and matched raised-line pictures to the objects. Performance was best in experiment 1 for top views, with shorter response latencies than for side views, front views, or 3-D views with foreshortening. In a second experiment with blind participants, matching accuracy was not significantly affected by prior visual experience, but speed advantages were found for top views, with 3-D views also yielding better matching accuracy than side views. There were no performance advantages for pictures of objects with a constant cross section in the vertical axis. The early-blind participants had lower performance for side and frontal views. The objects were rotated to oblique orientations in experiment 3. Early-blind subjects performed worse than the other subjects given object rotation. Visual experience with pictures of objects at many angles could facilitate identification at oblique orientations. In experiment 5 with blindfolded sighted subjects, tangible pictures were used as targets and as choices. The results yielded superior overall performance for 3-D views (mean, M = 74% correct) and much lower matching accuracy for top views as targets ( M = 58% correct). Performance was highest when the target and matching viewpoint were identical, but 3-D views ( M = 96% correct) were still far better than top views. The accuracy advantage of the top views also disappeared when more complex objects were tested in experiment 6. Alternative theoretical implications of the results are discussed.

Implicit and Explicit Memory for Haptically Experienced Two-Dimensional Patterns

Recent research with visual objects has delineated important representational differences between memory measures that tap identification (implicit tests) and measures that require episodic recognition (explicit tests) We investigated whether these differences reflect a fundamental architecture for the representation of object information in memory In the present experiment, we contrasted identification and episodic recognition for haptically presented two-dimensional patterns Haptic identification was not affected by elaborative processing at study, whereas haptic episodic recognition was enhanced by elaborative processing This finding suggests important similarities in the organization of object information in the visual and haptic modalities

Haptic-2D: A new haptic test battery assessing the tactual abilities of sighted and visually impaired children and adolescents with two-dimensional raised materials

To fill an important gap in the psychometric assessment of children and adolescents with impaired vision, we designed a new battery of haptic tests, called Haptic-2D, for visually impaired and sighted individuals aged five to 18 years. Unlike existing batteries, ours uses only two-dimensional raised materials that participants explore using active touch. It is composed of 11 haptic tests, measuring scanning skills, tactile discrimination skills, spatial comprehension skills, short-term tactile memory, and comprehension of tactile pictures. We administered this battery to 138 participants, half of whom were sighted (n=69), and half visually impaired (blind, n=16; low vision, n=53). Results indicated a significant main effect of age on haptic scores, but no main effect of vision or Age × Vision interaction effect. Reliability of test items was satisfactory (Cronbach's alpha, α=0.51-0.84). Convergent validity was good, as shown by a significant correlation (age partialled out) between total haptic scores and scores on the B101 test (rp=0.51, n=47). Discriminant validity was also satisfactory, as attested by a lower but still significant partial correlation between total haptic scores and the raw score on the verbal WISC (rp=0.43, n=62). Finally, test-retest reliability was good (rs=0.93, n=12; interval of one to two months). This new psychometric tool should prove useful to practitioners working with young people with impaired vision.

Contextual cueing: implicit memory of tactile context facilitates tactile search

In visual search, participants detect and subsequently discriminate targets more rapidly when these are embedded in repeatedly encountered distractor arrangements, an effect termed contextual cueing (Chun & Jiang Cognitive Psychology, 36, 28–71, 1998). However, whereas previous studies had explored contextual cueing exclusively in visual search, in the present study we examined the effect in tactile search using a novel tactile search paradigm. Participants were equipped with vibrotactile stimulators attached to four fingers on each hand. A given search array consisted of four stimuli (i.e., two items presented to each hand), with the target being an odd-one-out feature singleton that differed in frequency (Exps. 1 and 2) or waveform (Exp. 3) from the distractor elements. Participants performed a localization (Exps. 1 and 2) or discrimination (Exp. 3) task, delivering their responses via foot pedals. In all three experiments, reaction times were faster when the arrangement of distractor fingers predicted the target finger. Furthermore, participants were unable to explicitly discriminate repeated from nonrepeated tactile configurations (Exps. 2 and 3). This indicates that the tactile modality can mediate the formation of configural representations and use these representations to guide tactile search.

Simultaneous Sketching Aids the Haptic Identification of Raised Line Drawings

Haptically identifying raised line drawings is difficult. We investigated whether a major component of this difficulty lies in acquiring, integrating, and maintaining shape information from touch. Wijntjes, van Lienen, Verstijnen, and Kappers reported that drawings which participants had failed to identify by touch alone could often subsequently be named if they were sketched. Thus, people sometimes needed to externalize haptically acquired information by making a sketch in order to be able to use it. We extended Wijntjes et al.'s task and found that sketching while touching improved drawing identification even more than sketching after touching, but only if people could see their sketches. Our results suggest that the slow, serial nature of information acquisition seriously hampers the haptic identification of raised line drawings relative to visually identifying line drawings. Simultaneous sketching may aid identification by reducing the burden on working memory and by helping to guide haptic exploration. This conclusion is consistent with the finding reported by Lawson and Bracken that 3-D objects are much easier to identify haptically than raised line drawings since, unlike for vision, simultaneously extracting global shape information is much easier haptically for 3-D stimuli than for line drawings.

Haptic Object Recognition is View-Independent in Early Blind but not Sighted People

Object recognition, whether visual or haptic, is impaired in sighted people when objects are rotated between learning and test, relative to an unrotated condition, that is, recognition is view-dependent. Loss of vision early in life results in greater reliance on haptic perception for object identification compared with the sighted. Therefore, we hypothesized that early blind people may be more adept at recognizing objects despite spatial transformations. To test this hypothesis, we compared early blind and sighted control participants on a haptic object recognition task. Participants studied pairs of unfamiliar three-dimensional objects and performed a two-alternative forced-choice identification task, with the learned objects presented both unrotated and rotated 180° about they-axis. Rotation impaired the recognition accuracy of sighted, but not blind, participants. We propose that, consistent with our hypothesis, haptic view-independence in the early blind reflects their greater experience with haptic object perception.

Manual preferences for visually- and haptically-guided grasping

Studies have shown that individuals exhibit a right-hand preference for grasping during visually-guided tasks. Recently, we have found that when vision is occluded right-hand preference decreases dramatically. It remains unknown however, if this decrease is a result of visual occlusion or the effects of relying only on haptic feedback. Therefore, in the present study, we sought to explore the contributions of vision and haptics (separately and in conjunction) to hand preference for grasping. Right- and left-handed individuals were tested on a block building task under four different visual and haptic conditions: 1) vision/normal haptic feedback (V/H), 2) no vision/normal haptic feedback (NV/H), 3) vision/constrained haptic feedback (V/Constrained-H), and 4) no vision/constrained haptic feedback (NV/Constrained-H). Vision was occluded using a blindfold and haptic feedback was constrained by asking participants to wear textured gloves. Right-handed individuals displayed a right-hand preference when vision was available (V/H and V/Constrained-H groups), but this preference was much greater when haptic feedback was constrained (V/Constrained-H group). When vision was occluded and haptic feedback was used to complete the task (NV/H) no hand preference was found. Finally hand preference was similar between the V/H and the NV/Constrained-H groups. For left-handed individuals, no differences in hand use were found between the different sensory groups, but the NV/H group showed a clear left-hand preference for haptically-guided grasping. The results suggest that haptics plays an important role in hand preference for grasping. Furthermore, they support a left-hand/right-hemisphere specialization for haptically-guided grasping (regardless of handedness) and a right-hand/left-hemisphere specialization for visually-guided grasping (at least in right-handed individuals).

Designing Haptic Assistive Technology for Individuals Who Are Blind or Visually Impaired

This paper considers issues relevant for the design and use of haptic technology for assistive devices for individuals who are blind or visually impaired in some of the major areas of importance: Braille reading, tactile graphics, orientation and mobility. We show that there is a wealth of behavioral research that is highly applicable to assistive technology design. In a few cases, conclusions from behavioral experiments have been directly applied to design with positive results. Differences in brain organization and performance capabilities between individuals who are "early blind" and "late blind" from using the same tactile/haptic accommodations, such as the use of Braille, suggest the importance of training and assessing these groups individually. Practical restrictions on device design, such as performance limitations of the technology and cost, raise questions as to which aspects of these restrictions are truly important to overcome to achieve high performance. In general, this raises the question of what it means to provide functional equivalence as opposed to sensory equivalence.

Tactile search for change has less memory than visual search for change

Haptic perception of a 2D image is thought to make heavy demands on working memory. During active exploration, humans need to store the latest local sensory information and integrate it with kinesthetic information from hand and finger locations in order to generate a coherent perception. This tactile integration has not been studied as extensively as visual shape integration. In the current study, we compared working-memory capacity for tactile exploration to that of visual exploration as measured in change-detection tasks. We found smaller memory capacity during tactile exploration (approximately 1 item) compared with visual exploration (2-10 items). These differences generalized to position memory and could not be attributed to insufficient stimulus-exposure durations, acuity differences between modalities, or uncertainty over the position of items. This low capacity for tactile memory suggests that the haptic system is almost amnesic when outside the fingertips and that there is little or no cross-position integration.

Perception of tactile graphics: embossings versus cutouts

Graphical information, such as illustrations, graphs, and diagrams, are an essential complement to text for conveying knowledge about the world. Although graphics can be communicated well via the visual modality, conveying this information via touch has proven to be challenging. The lack of easily comprehensible tactile graphics poses a problem for the blind. In this paper, we advance a hypothesis for the limited effectiveness of tactile graphics. The hypothesis contends that conventional graphics that rely upon embossings on two-dimensional surfaces do not allow the deployment of tactile exploratory procedures that are crucial for assessing global shape. Besides potentially accounting for some of the shortcomings of current approaches, this hypothesis also serves a prescriptive purpose by suggesting a different strategy for conveying graphical information via touch, one based on cutouts. We describe experiments demonstrating the greater effectiveness of this approach for conveying shape and identity information. These results hold the potential for creating more comprehensible tactile drawings for the visually impaired while also providing insights into shape estimation processes in the tactile modality.

Grasping without sight: insights from the congenitally blind

We reach for and grasp different sized objects numerous times per day. Most of these movements are visually-guided, but some are guided by the sense of touch (i.e. haptically-guided), such as reaching for your keys in a bag, or for an object in a dark room. A marked right-hand preference has been reported during visually-guided grasping, particularly for small objects. However, little is known about hand preference for haptically-guided grasping. Recently, a study has shown a reduction in right-hand use in blindfolded individuals, and an absence of hand preference if grasping was preceded by a short haptic experience. These results suggest that vision plays a major role in hand preference for grasping. If this were the case, then one might expect congenitally blind (CB) individuals, who have never had a visual experience, to exhibit no hand preference. Two novel findings emerge from the current study: first, the results showed that contrary to our expectation, CB individuals used their right hand during haptically-guided grasping to the same extent as visually-unimpaired (VU) individuals did during visually-guided grasping. And second, object size affected hand use in an opposite manner for haptically- versus visually-guided grasping. Big objects were more often picked up with the right hand during haptically-guided, but less often during visually-guided grasping. This result highlights the different demands that object features pose on the two sensory systems. Overall the results demonstrate that hand preference for grasping is independent of visual experience, and they suggest a left-hemisphere specialization for the control of grasping that goes beyond sensory modality.