Toward a computational model of constraint-driven exploration and haptic object identification

Prior visual experience increases children's use of effective haptic exploration strategies in audio-tactile sound-shape correspondences

Sound-shape correspondence refers to the preferential mapping of information across the senses, such as associating a nonsense word like bouba with rounded abstract shapes and kiki with spiky abstract shapes. Here we focused on audio-tactile (AT) sound-shape correspondences between nonsense words and abstract shapes that are felt but not seen. Despite previous research indicating a role for visual experience in establishing AT associations, it remains unclear how visual experience facilitates AT correspondences. Here we investigated one hypothesis: seeing the abstract shapes improve haptic exploration by (a) increasing effective haptic strategies and/or (b) decreasing ineffective haptic strategies. We analyzed five haptic strategies in video-recordings of 6- to 8-year-old children obtained in a previous study. We found the dominant strategy used to explore shapes differed based on visual experience. Effective strategies, which provide information about shape, were dominant in participants with prior visual experience, whereas ineffective strategies, which do not provide information about shape, were dominant in participants without prior visual experience. With prior visual experience, poking-an effective and efficient strategy-was dominant, whereas without prior visual experience, uncategorizable and ineffective strategies were dominant. These findings suggest that prior visual experience of abstract shapes in 6- to 8-year-olds can increase the effectiveness and efficiency of haptic exploration, potentially explaining why prior visual experience can increase the strength of AT sound-shape correspondences.

Multisensory Texture Perception in Individuals with Williams Syndrome

The sensory profile of people with Williams syndrome (WS) is characterised by atypical visual and auditory perceptions that affect their daily lives and learning. However, no research has been carried out on the haptic perception, in particular in multisensory (visual and haptic) situations. The aim of this study was to evaluate the communication of texture information from one modality to the other in people with WS. Children and adults with WS were included, as well as typically developing (TD) participants matched on chronological age (TD-CA), and TD children matched on mental age (TD-MA). All participants (N = 69) completed three matching tasks in which they had to compare two fabrics (same or different): visual, haptic and visuo-haptic. When the textures were different, the haptic and visual performances of people with WS were similar to those of TD-MA participants. Moreover, their visuo-haptic performances were lower than those of the two TD groups. These results suggest a delay in the acquisition of multisensory transfer abilities in individuals with WS. A positive link between MA and visual and visuo-haptic abilities only in people with WS suggests that they could benefit from an early intervention to develop their abilities to process and transfer multisensory information.

Even distorted felt size engenders visual-haptic integration

Intersensory interactions predicted by the sensory precision hypothesis have been infrequently examined by distorting the reliability of size perception by touch. Consequently, participants were asked to see one size and manually feel another unseen size either with bare fingers or with fingers sleeved in rigid tubes to decrease the precision of touch. Their subsequent visual estimates of the perceived size favored the more precise modality. Experiment 1 (N = 46) varied the intersensory discordance to examine whether the estimate arose from trivial response biases or from perceptual binding effects. Experiment 2 (N = 32) examined the presence of the perceptual effect in the absence of discordant sensory cues. Results favored a perceptual interpretation because the haptic and visual cues merged regardless of the discordance amount only when the stimulation arose from separate sources. The observed interaction between touch imprecision and visual bias is consistent with computational models of optimal perception.

Frequency of Switching Touching Mode Reflects Tactile Preference Judgment

We can judge affective aspects of objects by actively exploring them with our hands. Previous studies have mainly focused on how the physical properties of an object’s surface affect tactile preference evaluations. However, despite the widely accepted notion that the participant’s strategy has a great impact on how they explore an object, there is a lack of investigations of hand motion during preference judgment and its impact on preference rating. This paper recruits the recurrence plot technique to illustrate the temporal dynamics of explorative hand motion. In an experiment, participants were asked to freely explore the surface of tactile stimuli and rate their tactile preference for them. The temporal dynamics of finger velocity and force were visualized and characterized by using recurrence quantification analysis. We found correlations between preference ratings and recurrence features that represent the temporal dynamics of explorative hand motion, in addition to correlations between preference ratings and conventional time-averaged features (e.g., averaged finger velocity). One unique feature that correlated with preference ratings was TREND, which represents to what extent similar motion patterns repeatedly occur. The results of a subsidiary analysis supported the possibility that the TREND difference can be interpreted as the frequency of switching touching modes (e.g., stroking and pushing motions). Taken together, these results suggest that participants tend to perform the same hand motion repeatedly for preferable objects, while they tend to combine different touching modes for less preferable objects. They also indicate that the recurrence plot scheme is a promising way to extract humans’ strategies for tactile exploration.

A Theoretical Framework of Haptic Processing in Automotive User Interfaces and Its Implications on Design and Engineering

Driving a car is a highly visual task. Despite the trend towards increased driver assistance and autonomous vehicles, drivers still need to interact with the car for both driving and non-driving relevant tasks, at times simultaneously. The often-resulting high cognitive load is a safety issue, which can be addressed by providing the driver with alternative feedback modalities, such as haptics. Recent trends in the automotive industry are moving towards the seamless integration of control elements through touch-sensitive surfaces. Psychological knowledge on optimally utilizing haptic technologies remains limited. The literature on automotive haptic feedback consists mainly of singular findings without putting them into a broader user context with respect to haptic design of interfaces. Moreover, haptic feedback has primarily been limited to the confirmation of control actions rather than the searching or finding of control elements, the latter of which becomes particularly important considering the current trends. This paper presents an integrated framework on haptic processing in automotive user interfaces and provides guidelines for haptic design of user interfaces in car interiors.

Hand-Feel Touch Cues and Their Influences on Consumer Perception and Behavior with Respect to Food Products: A Review

There has been a great deal of research investigating intrinsic/extrinsic cues and their influences on consumer perception and purchasing decisions at points of sale, product usage, and consumption. Consumers create expectations toward a food product through sensory information extracted from its surface (intrinsic cues) or packaging (extrinsic cues) at retail stores. Packaging is one of the important extrinsic cues that can modulate consumer perception, liking, and decision making of a product. For example, handling a product packaging during consumption, even just touching the packaging while opening or holding it during consumption, may result in a consumer expectation of the package content. Although hand-feel touch cues are an integral part of the food consumption experience, as can be observed in such an instance, little has been known about their influences on consumer perception, acceptability, and purchase behavior of food products. This review therefore provided a better understanding about hand-feel touch cues and their influences in the context of food and beverage experience with a focus on (1) an overview of touch as a sensory modality, (2) factors influencing hand-feel perception, (3) influences of hand-feel touch cues on the perception of other sensory modalities, and (4) the effects of hand-feel touch cues on emotional responses and purchase behavior.

Haptic Exploratory Strategies and Children who are Blind and Have Additional Disabilities

This study of the haptic exploratory strategies used by nine children with visual impairments and additional disabilities when interacting with portable and freely manipulable objects found that a broader approach to assessment and analysis is required than is used with typically developing children. An “adaptive-tasks” approach is proposed as a basis for developing a framework within which to account for sources of variance in the developing haptic abilities of these children.

Imaging of electrical activity in small diameter fibers of the murine peripheral nerve with virally-delivered GCaMP6f

Current neural interfaces are hampered by lack of specificity and selectivity for neural interrogation. A method that might improve these interfaces is an optical peripheral nerve interface which communicates with individual axons via optogenetic reporters. To determine the feasibility of such an interface, we delivered the genetically encoded calcium indicator GCaMP6f to the mouse peripheral nerve by intramuscular injection of adenoassociated viral vector (AAV1) under the control of the CAG (chicken beta actin- cytomegalovirus hybrid promoter). Small diameter axons in the common peroneal nerve were transduced and demonstrated electrically inducible calcium transients ex vivo. Responses to single electrical stimuli were resolvable, and increasing the number of stimuli resulted in a monotonic increase in maximum fluorescence and a prolongation of calcium transient kinetics. This work demonstrates the viability of using a virally-delivered, genetically-encoded calcium indicator to read-out from peripheral nerve axons.

Haptics to improve task performance in people with disabilities: A review of previous studies and a guide to future research with children with disabilities

This review examines the studies most pertinent to the potential of haptics on the functionality of assistive robots in manipulation tasks for use by children with disabilities. Haptics is the fast-emerging science that studies the sense of touch concerning the interaction of a human and his/her environment; this paper particularly studies the human–machine interaction that happens through a haptic interface to enable touch feedback. Haptics-enabled user interfaces for assistive robots can potentially benefit children whose haptic exploration is impaired due to a disability in their infancy and throughout their childhood. A haptic interface can provide touch feedback and potentially contribute to an enhancement in perception of objects and overall ability to perform manipulation tasks. The intention of this paper is to review the research on the applications of haptics, exclusively focusing on attributes affecting task performance. A review of studies will give a retrospective insight into previous research with various disability populations, and inform potential limitations/challenges in research regarding haptic interfaces for assistive robots for use by children with disabilities.

The dominance of haptics over audition in controlling wrist velocity during striking movements

Skilled interactions with sounding objects, such as drumming, rely on resolving the uncertainty in the acoustical and tactual feedback signals generated by vibrating objects. Uncertainty may arise from mis-estimation of the objects’ geometry-independent mechanical properties, such as surface stiffness. How multisensory information feeds back into the fine-tuning of sound-generating actions remains unexplored. Participants (percussionists, non-percussion musicians, or non-musicians) held a stylus and learned to control their wrist velocity while repeatedly striking a virtual sounding object whose surface stiffness was under computer control. Sensory feedback was manipulated by perturbing the surface stiffness specified by audition and haptics in a congruent or incongruent manner. The compensatory changes in striking velocity were measured as the motor effects of the sensory perturbations, and sensory dominance was quantified by the asymmetry of congruency effects across audition and haptics. A pronounced dominance of haptics over audition suggested a superior utility of somatosensation developed through long-term experience with object exploration. Large interindividual differences in the motor effects of haptic perturbation potentially arose from a differential reliance on the type of tactual prediction error for which participants tend to compensate: vibrotactile force versus object deformation. Musical experience did not have much of an effect beyond a slightly greater reliance on object deformation in mallet percussionists. The bias toward haptics in the presence of crossmodal perturbations was greater when participants appeared to rely on object deformation feedback, suggesting a weaker association between haptically sensed object deformation and the acoustical structure of concomitant sound during everyday experience of actions upon objects.

Please Touch: Object Properties that Invite Touch

Touch has received increasing interest in marketing, given research indicating that contact with products influences evaluation and the tendency to purchase. However, little is known from the marketing or psychophysical literature about visible attributes of objects that elicit touch for hedonic purposes. In these studies, participants rated the tendency of pictured objects to invite touch, or "touch-ability." Rated touch-ability varied reliably with structural attributes of objects, and the structural influences were distinct from those on other ratings such as attractiveness and apparent expense. Although the trends varied across object sets, touch-ability generally declined as surface textures became markedly rough and shape complexity became extreme. Holding stimulus factors constant, touch-ability also varied with the specific hand movements that were anticipated. Finally, mean touch-ability ratings were correlated across participants with the "Need for Touch" scale, which measures an individual's tendency to touch products. The studies point to touch-ability as a potential factor that might be incorporated into product design.

Mediating haptic exploratory strategies in children who have visual impairment and intellectual disabilities

This article provides a synthesis of literature pertaining to the development of haptic exploratory strategies in children who have visual impairment and intellectual disabilities. The information received through such strategies assumes particular significance for these children, given the restricted information available through their visual modality, often in combination with additional sensory and/or physical impairments. The literature reviewed from early child development highlights the importance of independent activity in the development of exploratory strategies, as well as the pivotal role of vision in 'mediating' information received through the haptic modality. In translating these findings to children who have visual impairment and intellectual disabilities, the role of the child's learning partner assumes greater significance in ensuring that haptic information is appropriately 'mediated' to meet the child's individual needs. The implications for developing appropriate developmentally paced intervention approaches are considered. A framework is outlined that seeks to account for the role of the child's adult partner in mediating haptic learning experiences to ensure they are appropriately structured and progressive.

Surface texture can bias tactile form perception

The sense of touch is believed to provide a reliable perception of the object's properties; however, our tactile perceptions could be illusory at times. A recently reported tactile illusion shows that a raised form can be perceived as indented when it is surrounded by textured areas. This phenomenon suggests that the form perception can be influenced by the surface textures in its adjacent areas. As perception of texture and that of form have been studied independently of each other, the present study examined whether textures, in addition to the geometric edges, contribute to the tactile form perception. We examined the perception of the flat and raised contact surface (3.0 mm width) with various heights (0.1, 0.2, 0.3 mm), which had either textured or non-textured adjacent areas, under the static, passive and active touch conditions. Our results showed that texture decreased the raised perception of the surface with a small height (0.1 mm) and decreased the flat perception of the physically flat surface under the passive and active touch conditions. We discuss a possible mechanism underlying the effect of the textures on the form perception based on previous neurophysiological findings.

“Abnormal” movements: What are they reflections of?

We agree with Latash & Anson that therapeutic approaches should be directed toward solving the underlying problem, not toward adapting the abnormal to normal behaviour. The fundamental obstacle, however, is that doing so requires a solution of the “equivalence problem” in movement control.

Defective preprogramming does not account for the clinical deficits of Parkinson's disease

Latash & Anson argue that changed motor patterns should not be viewed as pathological. Instead, they should be viewed as adaptations to a primary deficit. We argue that the evidence shows: (1) bradykinesia is not an adaptation to a different primary deficit, and (2) bradykinetic movements are not “normal” slowed movements but, to the contrary, bradykinesia is part of the pathophysiology of Parkinson's disease.

Generic mechanisms of coordination in special populations

We show that left-handers can be considered as a “special” population. We indicate that the asymmetries in performance exhibited by left-handers are due to a basic asymmetry in the underlying coordination dynamics that constrains bimanual coordination. In contrast to the claims of Latash & Anson, we argue that considerable knowledge has been gained regarding the essential equations of motion that govern biological coordination.

“Normal” is not the issue: It is “effective” goal attainment that counts

Adaptive motor patterns that emerge after a neuromusculoskeletal lesion reflect not only the primary lesion, but also the dynamic characteristics of the musculoskeletal linkage and environment in which the action is performed. Although these patterns may be optimal, they may also be ineffective as goal-directed actions; effectiveness may only be regained if training addresses primary deficits and ensures practice without gross biomechanical adaptations.

The goal of treatment for motor impairment is not to “normalize” but to “functionalize” through facilitative modulation and enabling context

Adaptations occurring in the central nervous system (CNS) in the presence of pathology are not uniformly “good” for the organism when viewed in a functional context. A functional reordering of CNS priorities can be produced by allowing restructuring of the external context or through modification of neuromuscular physiology or anatomy designed to reduce the inherent restriction of functional movement in upper motor neuron syndrome. In fact, volitional control can often be “unmasked” through such interventions. Therapeutic interventions should not be directed toward “normalization” of motor patterns but should permit a functional reordering of CNS priorities that would otherwise not be possible.

What are “normal movements” in atypical populations?

Redundancy of the motor control system is an important feature that gives the central control structures options for solving everyday motor problems. The choice of particular control patterns is based on priorities (coordinative rules) that are presently unknown. Motor patterns observed in unimpaired young adults reflect these priorities. We hypothesize that under certain atypical conditions, which may include disorders in perception of the environment and in decision making, structural or biochemical changes within the central nervous system (CNS), and/or structural changes of the effectors, the central nervous system may reconsider its priorities. A new set of priorities will reflect the current state of the system and may lead to different patterns of voluntary movement. Under such conditions, changed motor patterns should be considered not pathological but rather adaptive to a primary disorder and may even be viewed as optimal for a given state of the system of movement production. Therapeutic approaches should not be directed toward restoring the motor patterns to as close to “normal” as possible but rather toward resolving the original underlying problem. We illustrate this approach using, as examples, movements in amputees, in patients with Parkinson's disease, in patients with dystonia, and in persons with Down syndrome.

What are “normal movements” inanypopulation?

It is the contention of Latash & Anson (L&A) that in atypical populations, such as those with cognitive, central neurological, or peripheral disorders, the central nervous system (CNS) is capable of producing more effective, though often less “normal,” movement patterns ifleft to its own devices. It is the aim of this commentary to extend their argument to other populations by pointing out the many parallels with development of movement patterns in sport.

Should stereotypic movement synergies in hemiparetic patients be considered adaptive?

The motor deficits observed in patients following some lesions of the central nervous system may be viewed as falling at one extreme of the continuum of possible motor behaviours. They are usually associated with an impaired ability to select and control specific movements from the available repertoire. Such movements may not be viewed as abnormal. However, it is unlikely that the primary motor deficits can all be considered adaptive.

“Constraint” versus “choice” in preferred movement patterns

Movement patterns in impaired gait are not selected by a smart central nervous system (CNS), but arise by virtue of mutual constraints of task, limitations in personal dynamics available for the task, and optimality criteria. An oscillatory model of gait cycle is presented that exemplifies this control and coordination scheme. Preferred gait patterns may be based on physical principles rather than CNS “coordinative rules.”

Developmental “movement disorders” and problem solving

Normal infants show a wide range of “atypical” movements. These, like the movements of atypical and normal adults, are best characterized as solutions to motor problems. Motor patterns alone may not be precise indicators of neurological status.

Is motor pathology associated with setting new CNS priorities or with increased difficulty in overcoming or suppressing preexisting CNS priorities?

Whereas Latash & Anson (L&A) have underscored the rearrangement or setting of new priorities, our primary focus is on preexisting central nervous system (CNS) priorities that become even more prevalent and intrusive under pathological conditions. The adaptations observed in the disordered motor system can often be understood against the backdrop of these primary CNS constraints. Even though this perspective has not been specifically addressed in the target article, we consider it complementary and not necessarily in opposition to L&A's primary thesis.

Haptic perception after a change in hand size

When we actively explore with our hands (haptics), the brain must combine diverse sensory signals into a unified percept. Presumably, a key role is played by an internal model of the body, but the processes responsible for forming and maintaining such a model are poorly understood. We used a multisensory illusion to alter the internal model of one's hand by using fake hands smaller or larger than the participant's hand. After such brief illusion we observed a change in haptically perceived object size. These results demonstrate that the brain rapidly recalibrates haptic signals using an internal model of the hand and that this model is multisensory rather than merely visual or memory-based.

The haptic abilities of the human newborn

Abstract. How do human babies gain knowledge by touch? This review presents information on the haptic abilities of human newborns and preterm infants which sheds light on this question. How do full-term and preterm newborns perceive information and form a perceptual representation of objects extracted from the hands alone? How do full-term newborns transfer this information to vision in an intermodal process? Using a habituation/dishabituation procedure, experiments have revealed that preterm and full-term newborns are able to discriminate object shapes in the manual, as well as in the visual modalities for full-term newborns. These abilities are a prerequisite for the establishment of relations between haptic and visual sensory information in cross-modal transfer tasks. We discuss several experiments performed using an intersensory successive preference procedure which provide evidence for cross-modal recognition from touch to vision from birth in full-term newborns. The links, however, are limited, partial, and not always reciprocal. We discuss these results in the light of recent theories on visual-haptic links.

The virtual haptic display: a device for exploring 2-D virtual shapes in the tactile modality

In order to understand better the processes involved in the perception of shape through touch, someelement of control is required over the nature of the shape presented to the hand and the presentation timing. To that end, we have developed a cost-effective, computer-controlled apparatus for presenting haptic stimuli using active touch, known as a virtual haptic display (VHD). The operational principle behind this device is that it translates black and white visual images into topographic, 2-D taxel (tactile pixel) arrays, along the same principle using in Braille letters. These taxels are either elevated or depressed at any one time representing white and black pixel colors of the visual image, respectively. To feel the taxels, the participant places their fingers onto a carriage which can be moved over the surface of the device to reveal a virtual shape. We conducted two experiments and the results show that untrained participants are able to recognize different, simple and complex, shapes using this apparatus. The VHD apparatus is therefore ideal at presenting 2-Dshapes through touch alone. Moreover,this device and its supporting software can also be used for presenting computer-controlled stimuli in cross-modal experiments.

WEIGHT PERCEPTION IN NEONATE INFANTS

A recent research revealed the capacity of the newborn to haptically detect, the weight of an object [12]. In this research, we tried to determine the means by which newborn infants, not having yet exploratory procedures, are able to treat this object property. We support the assumption that tactile perceptive capacities of the newborn infants derive from a fundamental property of the sensorimotor system: its primary variability. After a period of habituation with a heavy or light object, an object of new weight is presented (period test). Three parameters of the sensorimotor activity were analyzed during these two periods: holding times, amplitude of the pressure exerted on the object and frequency of the pressure. Analyses of these parameters demonstrate the neonate's capacity to discriminate objects weight.

SOMESTHESIS

In this review we focus on the perceptual and psychophysical aspects of somesthesis, although some information on neurophysiological aspects will be included as well; we look primarily at studies that have appeared since 1988. In the section on touch, we cover peripheral sensory mechanisms and several topics related to spatial and temporal pattern perception, specifically measures of spatial sensitivity, texture perception with particular emphasis on perceived roughness, complex spatial-temporal patterns, and the use of touch as a possible channel of communication. Other topics under this section include the effects of attention on processing tactile stimuli, cortical mechanisms, and the effects of aging on sensitivity. We also deal with thermal sensitivity and some aspects of haptics and kinesthesis. In the section on pain, we review work on the gate-control theory, sensory fibers, and higher neural organization. In addition, studies on central neurochemical effects and psychophysics of pain are examined.

Cognitive salience of haptic object properties: role of modality-encoding bias

The influence of modality-encoding bias on the relative importance ('cognitive salience') of object shape, size, and material, with the last determined by weight and thermal variations, was examined. Experiment 1 confirmed that for these stimulus objects all five properties were very accessible haptically, as measured by the time to identify the property level of each designated property; however, observers were still generally faster for geometric than material properties. In experiment 2, the influence of modality-encoding bias on cognitive salience was assessed by using a task involving free sorting by similarity. As predicted, modality-encoding bias strongly influenced cognitive salience. Observers favoured sorting by material under haptic- bias instructions, and three-dimensional geometric properties (especially shape) under visual-bias instructions. Videotaped hand movements indicated that modality-encoding biases reflect long-term knowledge of the relative speed and precision of manual exploration patterns, rather than exploration of the current set of objects.