Haptic exploratory behavior during object discrimination: a novel automatic annotation method

In order to acquire information concerning the geometry and material of handheld objects, people tend to execute stereotypical hand movement patterns called haptic Exploratory Procedures (EPs). Manual annotation of haptic exploration trials with these EPs is a laborious task that is affected by subjectivity, attentional lapses, and viewing angle limitations. In this paper we propose an automatic EP annotation method based on position and orientation data from motion tracking sensors placed on both hands and inside a stimulus. A set of kinematic variables is computed from these data and compared to sets of predefined criteria for each of four EPs. Whenever all criteria for a specific EP are met, it is assumed that that particular hand movement pattern was performed. This method is applied to data from an experiment where blindfolded participants haptically discriminated between objects differing in hardness, roughness, volume, and weight. In order to validate the method, its output is compared to manual annotation based on video recordings of the same trials. Although mean pairwise agreement is less between human-automatic pairs than between human-human pairs (55.7% vs 74.5%), the proposed method performs much better than random annotation (2.4%). Furthermore, each EP is linked to a specific object property for which it is optimal (e.g., Lateral Motion for roughness). We found that the percentage of trials where the expected EP was found does not differ between manual and automatic annotation. For now, this method cannot yet completely replace a manual annotation procedure. However, it could be used as a starting point that can be supplemented by manual annotation.

Target contact and exploration strategies in haptic search

In a haptic search task, one has to detect the presence of a target among distractors using the sense of touch. A salient target can be detected faster than a non-salient target. However, little is known about the exploration strategies that are used, especially in 3D search tasks where items are held in the hand. In this study, we investigated which parts of the hand were used to contact the target and which strategies were performed. Blindfolded participants performed search tasks in four conditions, where the targets differed in relevant property and saliency. The positions of the target and the hand were tracked during exploration. It was found that target saliency had a large effect on the use of the hand parts and the strategies. In the non-salient target conditions, the fingers, especially the thumb, contacted the target more often than in the salient target conditions. This could also be seen in the strategies, where the thumb was used to explore the items in a serial way by moving them in the hand or touching them individually. In the salient target conditions, more parallel strategies like grasping or shuffling of the items in the hand were used.

Haptic discrimination of distance

While quite some research has focussed on the accuracy of haptic perception of distance, information on the precision of haptic perception of distance is still scarce, particularly regarding distances perceived by making arm movements. In this study, eight conditions were measured to answer four main questions, which are: what is the influence of reference distance, movement axis, perceptual mode (active or passive) and stimulus type on the precision of this kind of distance perception? A discrimination experiment was performed with twelve participants. The participants were presented with two distances, using either a haptic device or a real stimulus. Participants compared the distances by moving their hand from a start to an end position. They were then asked to judge which of the distances was the longer, from which the discrimination threshold was determined for each participant and condition. The precision was influenced by reference distance. No effect of movement axis was found. The precision was higher for active than for passive movements and it was a bit lower for real stimuli than for rendered stimuli, but it was not affected by adding cutaneous information. Overall, the Weber fraction for the active perception of a distance of 25 or 35 cm was about 11% for all cardinal axes. The recorded position data suggest that participants, in order to be able to judge which distance was the longer, tried to produce similar speed profiles in both movements. This knowledge could be useful in the design of haptic devices.

Digit forces bias sensorimotor transformations underlying control of fingertip position

Humans are able to modulate digit forces as a function of position despite changes in digit placement that might occur from trial to trial or when changing grip type for object manipulation. Although this phenomenon is likely to rely on sensing the position of the digits relative to each other and the object, the underlying mechanisms remain unclear. To address this question, we asked subjects (n = 30) to match perceived vertical distance between the center of pressure (CoP) of the thumb and index finger pads (d_y) of the right hand (“reference” hand) using the same hand (“test” hand). The digits of reference hand were passively placed collinearly (d_y = 0 mm). Subjects were then asked to exert different combinations of normal and tangential digit forces (F_n and F_tan, respectively) using the reference hand and then match the memorized d_y using the test hand. The reference hand exerted F_tan of thumb and index finger in either same or opposite direction. We hypothesized that, when the tangential forces of the digits are produced in opposite directions, matching error (1) would be biased toward the directions of the tangential forces; and (2) would be greater when the remembered relative contact points are matched with negligible digit force production. For the test hand, digit forces were either negligible (0.5–1 N, 0 ± 0.25 N; Experiment 1) or the same as those exerted by the reference hand (Experiment 2).Matching error was biased towards the direction of digit tangential forces: thumb CoP was placed higher than the index finger CoP when thumb and index finger F_tan were directed upward and downward, respectively, and vice versa (p < 0.001). However, matching error was not dependent on whether the reference and test hand exerted similar or different forces. We propose that the expected sensory consequence of motor commands for tangential forces in opposite directions overrides estimation of fingertip position through haptic sensory feedback.

Parallel processing of shape and texture in haptic search

In a haptic search task, one has to determine the presence of a target among distractors. It has been shown that if the target differs from the distractors in two properties, shape and texture, performance is better than in both single-property conditions (Van Polanen, Bergmann Tiest, & Kappers, 2013). The search for a smooth sphere among rough cubical distractors was faster than both the searches for a rough sphere (shape information only) and for a smooth cube (texture information only). This effect was replicated in this study as a baseline. The main focus here was to further investigate the nature of this integration. It was shown that performance is better when the two properties are combined in a single target (smooth sphere), than when located in two separate targets (rough sphere and smooth cube) that are simultaneously present. A race model that assumes independent parallel processing of the two properties could explain the enhanced performance with two properties, but this could only take place effectively when the two properties were located in a single target.

Contact force and scanning velocity during active roughness perception

Haptic perception is bidirectionally related to exploratory movements, which means that exploration influences perception, but perception also influences exploration. We can optimize or change exploratory movements according to the perception and/or the task, consciously or unconsciously. This paper presents a psychophysical experiment on active roughness perception to investigate movement changes as the haptic task changes. Exerted normal force and scanning velocity are measured in different perceptual tasks (discrimination or identification) using rough and smooth stimuli. The results show that humans use a greater variation in contact force for the smooth stimuli than for the rough stimuli. Moreover, they use higher scanning velocities and shorter break times between stimuli in the discrimination task than in the identification task. Thus, in roughness perception humans spontaneously use different strategies that seem effective for the perceptual task and the stimuli. A control task, in which the participants just explore the stimuli without any perceptual objective, shows that humans use a smaller contact force and a lower scanning velocity for the rough stimuli than for the smooth stimuli. Possibly, these strategies are related to aversiveness while exploring stimuli.

Influence of shape on the haptic size aftereffect

Recently, we showed a strong haptic size aftereffect by means of a size bisection task: after adaptation to a large sphere, subsequently grasped smaller test spheres felt even smaller, and vice versa. In the current study, we questioned whether the strength of this aftereffect depends on shape. In four experimental conditions, we determined the aftereffect after adaptation to spheres and tetrahedra and subsequent testing also with spheres and tetrahedra. The results showed a clear influence of shape: the haptic aftereffect was much stronger if adaptation and test stimuli were identical in shape than if their shapes were different. Therefore, it would be more appropriate to term such aftereffects haptic shape-size aftereffects, as size alone could not be the determining factor. This influence of shape suggests that higher cortical areas are involved in this aftereffect and that it cannot be due to adaptation of peripheral receptors. An additional finding is that the geometric property or combination of properties participants use in the haptic size bisection task varies widely over participants, although participants themselves are quite consistent.

Identifying haptic exploratory procedures by analyzing hand dynamics and contact force

Haptic exploratory procedures (EPs) are prototypical hand movements that are linked to the acquisition of specific object properties. In studies of haptic perception, hand movements are often classified into these EPs. Here, we aim to investigate several EPs in a quantitative manner to understand how hand dynamics and contact forces differ between them. These dissimilarities are then used to construct an EP identification model capable of discriminating between EPs based on the index finger position and contact force. The extent to which the instructed EPs were distinct, repeatable, and similar across subjects was confirmed by showing that more than 95 percent of the analyzed trials were classified correctly. Finally, the method is employed to investigate haptic exploratory behavior during similarity judgments based on several object properties. It seems that discrimination based on material properties (hardness, roughness, and temperature) yields more consistent classification results compared to discrimination based on the acquisition of shape information.

Anisotropy in the haptic perception of force direction and magnitude

Although force-feedback devices are already being used, the human ability to perceive forces has not been documented thoroughly. The haptic perception of force direction and magnitude has mostly been studied in discrimination tasks in the direction of gravity. In our study, the influence of physical force direction on haptic perception of force magnitude and direction was studied in the horizontal plane. Subjects estimated the direction and magnitude of a force exerted on their stationary hand. A significant anisotropy in perception of force magnitude and direction was found. Force direction data showed significant subject-dependent distortions at various physical directions. Normalized force magnitude data showed a consistent elliptical pattern, with its minor axis pointing roughly from the subject's hand to his/her shoulder. This pattern could be related to arm stiffness or manipulability patterns, which are also ellipse-shaped. These ellipses have an orientation consistent with the distortion measured in our study. So, forces in the direction of highest stiffness and lowest manipulability are perceived as being smaller. It therefore seems that humans possess a "sense of effort" rather than a "sense of force," which may be more useful in everyday life. These results could be useful in the design of haptic devices.

Integration and disruption effects of shape and texture in haptic search

In a search task, where one has to search for the presence of a target among distractors, the target is sometimes easily found, whereas in other searches it is much harder to find. The performance in a search task is influenced by the identity of the target, the identity of the distractors and the differences between the two. In this study, these factors were manipulated by varying the target and distractors in shape (cube or sphere) and roughness (rough or smooth) in a haptic search task. Participants had to grasp a bundle of items and determine as fast as possible whether a predefined target was present or not. It was found that roughness and edges were relatively salient features and the search for the presence of these features was faster than for their absence. If the task was easy, the addition of these features could also disrupt performance, even if they were irrelevant for the search task. Another important finding was that the search for a target that differed in two properties from the distractors was faster than a task with only a single property difference, although this was only found if the two target properties were non-salient. This means that shape and texture can be effectively integrated. Finally, it was found that edges are more beneficial to a search task than disrupting, whereas for roughness this was the other way round.

Superior spatial touch: improved haptic orientation processing in deaf individuals

The present study investigated haptic spatial orientation processing in deaf signers, hearing sign language interpreters, and hearing controls. Blindfolded participants had to set two bars parallel in the horizontal plane, with either a 2-s or a 10-s delay between inspection of the reference bar and the setting of the test bar. The deaf group outperformed the other two groups which did not differ from each other. Together these results indicate that deaf individuals can better identify the allocentric spatial coordinates of haptically inspected orientations. These results are discussed in terms of the possible neurocognitive consequences of auditory deprivation.

Aging and curvature discrimination from static and dynamic touch

Two experiments evaluated the ability of 30 older and younger adults to discriminate the curvature of simple object surfaces from static and dynamic touch. The ages of the older adults ranged from 66 to 85 years, while those of the younger adults ranged from 20 to 29 years. For each participant in both experiments, the minimum curvature magnitude needed to reliably discriminate between convex and concave surfaces was determined. In Experiment 1, participants used static touch to make their judgments of curvature, while dynamic touch was used in Experiment 2. When static touch was used to discriminate curvature, a large effect of age occurred (the thresholds were 0.67 & 1.11/m for the younger and older participants, respectively). However, when participants used dynamic touch, there was no significant difference between the ability of younger and older participants to discriminate curvature (the thresholds were 0.58 & 0.59/m for the younger and older participants, respectively). The results of the current study demonstrate that while older adults can accurately discriminate surface curvature from dynamic touch, they possess significant impairments for static touch.

Short term memory and working memory in blind versus sighted children

There is evidence that blind people may strengthen their memory skills to compensate for absence of vision. However, which aspects of memory are involved is open to debate and a developmental perspective is generally lacking. In the present study, we compared the short term memory (STM) and working memory (WM) of 10-year-old blind children and sighted children. STM was measured using digit span forward, name learning, and word span tasks; WM was measured using listening span and digit span backward tasks. The blind children outperformed their sighted peers on both STM and WM tasks. The enhanced capacity of the blind children on digit span and other STM tasks confirms the results of earlier research; the significantly better performance of the blind children relative to their sighted peers on verbal WM tasks is a new interesting finding. Task characteristics, including the verbal nature of the WM tasks and strategies used to perform these tasks, are discussed.

Bimanual integration of position and curvature in haptic perception

For small stimuli, it has been shown that subjects are very accurate in distinguishing a cylinder with an elliptical cross section from one with a circular cross section. In such a task, both curvature and length are integrated effectively. Large cylinders are explored differently: either by one hand or by two hands sliding over the surface. However, the same cues are available. We investigated the integration of position and curvature in unimanual and bimanual explorations. In Experiment 1, curved surfaces were presented as part of a horizontal cylinder with a cross section that was either a horizontally or a vertically elongated ellipse. We found that discrimination thresholds for unimanual exploration were significantly larger than for bimanual exploration. In Experiment 2, we found that position discrimination thresholds were independent of the type of exploration (unimanual or bimanual) and surprisingly also independent of the reference length. In Experiment 3, we found that discrimination thresholds for the position of the midsagittal plane were on an average lower than the position discrimination thresholds found in Experiment 2. From these findings, we conclude that the lower thresholds in Experiment 1 for bimanual exploration compared to unimanual exploration are due to the integration of curvature, not position or uncertainty of the midsagittal plane in unimanual exploration.

Genotype and phenotype of 101 dutch patients with congenital stationary night blindness

OBJECTIVE

To investigate the relative frequency of the genetic causes of the Schubert-Bornschein type of congenital stationary night blindness (CSNB) and to determine the genotype-phenotype correlations in CSNB1 and CSNB2.

DESIGN

Clinic-based, longitudinal, multicenter study.

PARTICIPANTS

A total of 39 patients with CSNB1 from 29 families and 62 patients with CSNB2 from 43 families.

METHODS

Patients underwent full ophthalmologic and electrophysiologic examinations. On the basis of standard electroretinograms (ERGs), patients were diagnosed with CSNB1 or CSNB2. Molecular analysis was performed by direct Sanger sequencing of the entire coding regions in NYX, TRPM1, GRM6, and GPR179 in patients with CSNB1 and CACNA1F and CABP4 in patients with CSNB2.

MAIN OUTCOME MEASURES

Data included genetic cause of CSNB, refractive error, visual acuity, nystagmus, strabismus, night blindness, photophobia, color vision, dark adaptation (DA) curve, and standard ERGs.

RESULTS

A diagnosis of CSNB1 or CSNB2 was based on standard ERGs. The photopic ERG was the most specific criterion to distinguish between CSNB1 and CSNB2 because it showed a "square-wave" appearance in CSNB1 and a decreased b-wave in CSNB2. Mutations causing CSNB1 were found in NYX (20 patients, 13 families), TRPM1 (10 patients, 9 families), GRM6 (4 patients, 3 families), and GPR179 (2 patients, 1 family). Congenital stationary night blindness 2 was primarily caused by mutations in CACNA1F (55 patients, 37 families). Only 3 patients had causative mutations in CABP4 (2 families). Patients with CSNB1 mainly had rod-related problems, and patients with CSNB2 had rod- and cone-related problems. The visual acuity on average was better in CSNB1 (0.30 logarithm of the minimum angle of resolution [logMAR]) than in CSNB2 (0.52 logMAR). All patients with CSNB1 and only 54% of the patients with CSNB2 reported night blindness. The dark-adapted threshold was on average more elevated in CSNB1 (3.0 log) than in CSNB2 (1.8 log). The 3 patients with CABP4 had a relative low visual acuity, were hyperopic, had severe nonspecific color vision defects, and had only 1.0 log elevated DA threshold.

CONCLUSIONS

Congenital stationary night blindness 1, despite different causative mutations, shows 1 unique CSNB1 phenotype. Congenital stationary night blindness 2 caused by mutations in CABP4 merely shows cone-related problems and therefore appears to be distinct from CSNB2 caused by mutations in CACNA1F.

FINANCIAL DISCLOSURE(S)

The author(s) have no proprietary or commercial interest in any materials discussed in this article.

Haptic perception

Fueled by novel applications, interest in haptic perception is growing. This paper provides an overview of the state of the art of a number of important aspects of haptic perception. By means of touch we can not only perceive quite different material properties, such as roughness, compliance, friction, coldness and slipperiness, but we can also perceive spatial properties, such as shape, curvature, size and orientation. Moreover, the number of objects we have in our hand can be determined, either by counting or subitizing. All these aspects will be presented and discussed in this paper. Although our intuition tells us that touch provides us with veridical information about our environment, the existence of prominent haptic illusions will show otherwise. Knowledge about haptic perception is interesting from a fundamental viewpoint, but it also is of eminent importance in the technological development of haptic devices. At the end of this paper, a few recent applications will be presented. WIREs Cogn Sci 2013, 4:357-374. DOI: 10.1002/wcs.1238 CONFLICT OF INTEREST: The authors have declared no conflicts of interest for this article. For further resources related to this article, please visit the WIREs website.

Assessment of night vision problems in patients with congenital stationary night blindness

Congenital Stationary Night Blindness (CSNB) is a retinal disorder caused by a signal transmission defect between photoreceptors and bipolar cells. CSNB can be subdivided in CSNB2 (rod signal transmission reduced) and CSNB1 (rod signal transmission absent). The present study is the first in which night vision problems are assessed in CSNB patients in a systematic way, with the purpose of improving rehabilitation for these patients. We assessed the night vision problems of 13 CSNB2 patients and 9 CSNB1 patients by means of a questionnaire on low luminance situations. We furthermore investigated their dark adapted visual functions by the Goldmann Weekers dark adaptation curve, a dark adapted static visual field, and a two-dimensional version of the “Light Lab”. In the latter test, a digital image of a living room with objects was projected on a screen. While increasing the luminance of the image, we asked the patients to report on detection and recognition of objects. The questionnaire showed that the CSNB2 patients hardly experienced any night vision problems, while all CSNB1 patients experienced some problems although they generally did not describe them as severe. The three scotopic tests showed minimally to moderately decreased dark adapted visual functions in the CSNB2 patients, with differences between patients. In contrast, the dark adapted visual functions of the CSNB1 patients were more severely affected, but showed almost no differences between patients. The results from the “2D Light Lab” showed that all CSNB1 patients were blind at low intensities (equal to starlight), but quickly regained vision at higher intensities (full moonlight). Just above their dark adapted thresholds both CSNB1 and CSNB2 patients had normal visual fields. From the results we conclude that night vision problems in CSNB, in contrast to what the name suggests, are not conspicuous and generally not disabling.

Evaluation of robotically controlled advanced endoscopic instruments

BACKGROUND

Advanced flexible endoscopes and instruments with multiple degrees of freedom enable physicians to perform challenging procedures such as the removal of large sections of mucosal tissue. However, these advanced endoscopes are difficult to control and require several physicians to cooperate.

METHODS

In this article, we present a robotic system that allows the physician to control an instrument in an intuitive way, using a haptic device. Performance with the robotic and conventional control methods were compared in a human subjects experiment. Subjects used both methods to tap a series of targets. They performed four trials while looking at the endoscopic monitor, and two trials while looking at the instrument directly.

RESULTS

Subjects were significantly faster using the robotic method, 54 s vs 164 s. Their performance in the second trial was significantly improved with respect to the first trial.

CONCLUSIONS

This study provides evidence that the robotic control method can be implemented to improve the performance of physicians using advanced flexible endoscopes.

Haptic discrimination and matching of viscosity

In three experiments, viscosity perception of liquids using the sense of touch was studied. The first two were discrimination experiments in which Weber fractions were determined for a number of viscosities spanning the range of what is encountered in daily life, and for two ways of perceiving viscosity (stirring with a spatula or with the index finger). For high viscosities, Weber fractions were around 0.3, whereas they increased for lower viscosities. For low viscosities, discrimination performance was much worse with the finger than with the spatula. In the third experiment, subjects matched liquids perceived with these two methods, which resulted in biases of around 80. Control experiments and force measurements were performed to find an explanation for these results. It was concluded that the relationship between perceived and physical viscosity is steeper for stirring liquids with a spatula than stirring with the finger.

Haptic search for hard and soft spheres

In this study the saliency of hardness and softness were investigated in an active haptic search task. Two experiments were performed to explore these properties in different contexts. In Experiment 1, blindfolded participants had to grasp a bundle of spheres and determine the presence of a hard target among soft distractors or vice versa. If the difference in compliance between target and distractors was small, reaction times increased with the number of items for both features; a serial strategy was found to be used. When the difference in compliance was large, the reaction times were independent of the number of items, indicating a parallel strategy. In Experiment 2, blindfolded participants pressed their hand on a display filled with hard and soft items. In the search for a soft target, increasing reaction times with the number of items were found, but the location of target and distractors appeared to have a large influence on the search difficulty. In the search for a hard target, reaction times did not depend on the number of items. In sum, this showed that both hardness and softness are salient features.

Auditory frequency discrimination in adults with dyslexia: a test of the anchoring hypothesis

PURPOSE

A recent hypothesis ascribes dyslexia to a perceptual anchoring deficit. Supporting results have so far been obtained only in children with dyslexia and additional learning difficulties, but the hypothesis has been argued to apply to all individuals with dyslexia.

METHOD

The authors measured auditory frequency discrimination thresholds in 54 young adults (native speakers of Dutch; 25 individuals with dyslexia; 29 age- and education-matched controls), with a 2-alternative forced-choice staircase paradigm. There were 2 conditions: 1 in which 1 tone in each pair had a fixed frequency across trials (standard), and 1 in which both tones varied in frequency (no standard). Presence of a standard stimulus allows the creation of a stimulus-specific representation in short-term-memory (perceptual anchor).

RESULTS

The standard condition yielded significantly lower discrimination thresholds in both controls and individuals with dyslexia; both groups benefited equally from the presence of an invariant stimulus.

CONCLUSION

There is no difference between this group of adults with dyslexia and controls in their capacity to form a perceptual anchor. The implication is that an anchoring deficit cannot be generalized to all cases of dyslexia.

Human perception of shape from touch

In this paper, I focus on the role of active touch in three aspects of shape perception and discrimination studies. First an overview is given of curvature discrimination experiments. The most prominent result is that first-order stimulus information (that is, the difference in attitude or slope over the stimulus) is the dominant factor determining the curvature threshold. Secondly, I compare touch under bimanual and two-finger performance with unimanual and one-finger performance. Consistently, bimanual or two-finger performance turned out to be worse. The most likely explanation for the former finding is that a loss of accuracy during intermanual comparisons is owing to interhemispheric relay. Thirdly, I address the presence of strong after-effects after just briefly touching a shape. These after-effects have been measured and studied in various conditions (such as, static, dynamic, transfer to other hand or finger). Combination of the results of these studies leads to the insight that there are possibly different classes of after-effect: a strong after-effect, caused by immediate contact with the stimulus, that does only partially transfer to the other hand, and one much less strong after-effect, caused by moving over the stimulus for a certain period, which shows a full transfer to other fingers.

Influence of Local Properties on the Perception of Global Object Orientation

In previous studies, the effect on perception of individual features such as curvature and edges have been studied with specifically designed stimuli. However, the effect of local properties on the perception of the global object has so far received little attention. In this study, cylinders with an elliptical cross section and rectangular blocks were used to investigate the effect and relative importance of curvature, change in curvature and edges, as local properties, on the ability of subjects to determine the orientation of the stimuli, which is a global property. We found that when curvature was present the threshold to determine the orientation was 43 percent lower than when curvature was absent. When, in addition, the change in curvature could be felt, the threshold was 37 percent lower than when only curvature could be felt. Finally, when edges were felt during exploration, the threshold increased by 46 percent compared to when the subjects were instructed to avoid the edges in the blocks. We conclude that the perception of curvature and change in curvature improve the performance of humans in perception of the whole shape, whereas edges, when not directly contributing to the task, disrupt performance.

Phase change materials and the perception of wetness

Phase change materials (PCMs) are increasingly incorporated in textiles in order to serve as a thermal buffer when humans change from a hot to a cold environment and the reverse. Due to the absence of wetness sensors in the skin, cooling of the skin may be perceived as a sensation of wetness instead of cold. In order to investigate if this phenomenon occurs when manipulating textiles, nine subjects were asked to touch or manipulate PCM-treated and untreated fabrics. In 75% of the cases, the subjects indicated that the treated material felt wetter than the untreated material independent of the way the textiles were manipulated. We conclude that incorporating PCMs in textiles may lead to a feeling of wetness which might be uncomfortable. Therefore, we recommend investigating a change in cooling properties to minimise this feeling.

PRACTITIONER SUMMARY

This article describes a psychophysical experiment into the sensation of wetness of textiles treated with phase change materials. It was found that in 75% of the cases, subjects found the treated fabric to feel wetter than the untreated. This may affect the comfort of wearing clothes made of these textiles.