The size-weight illusion induced through human echolocation

Object recognition via echoes: quantifying the crossmodal transfer of three-dimensional shape information between echolocation, vision, and haptics

Active echolocation allows blind individuals to explore their surroundings via self-generated sounds, similarly to dolphins and other echolocating animals. Echolocators emit sounds, such as finger snaps or mouth clicks, and parse the returning echoes for information about their surroundings, including the location, size, and material composition of objects. Because a crucial function of perceiving objects is to enable effective interaction with them, it is important to understand the degree to which three-dimensional shape information extracted from object echoes is useful in the context of other modalities such as haptics or vision. Here, we investigated the resolution of crossmodal transfer of object-level information between acoustic echoes and other senses. First, in a delayed match-to-sample task, blind expert echolocators and sighted control participants inspected common (everyday) and novel target objects using echolocation, then distinguished the target object from a distractor using only haptic information. For blind participants, discrimination accuracy was overall above chance and similar for both common and novel objects, whereas as a group, sighted participants performed above chance for the common, but not novel objects, suggesting that some coarse object information (a) is available to both expert blind and novice sighted echolocators, (b) transfers from auditory to haptic modalities, and (c) may be facilitated by prior object familiarity and/or material differences, particularly for novice echolocators. Next, to estimate an equivalent resolution in visual terms, we briefly presented blurred images of the novel stimuli to sighted participants (N = 22), who then performed the same haptic discrimination task. We found that visuo-haptic discrimination performance approximately matched echo-haptic discrimination for a Gaussian blur kernel σ of ~2.5°. In this way, by matching visual and echo-based contributions to object discrimination, we can estimate the quality of echoacoustic information that transfers to other sensory modalities, predict theoretical bounds on perception, and inform the design of assistive techniques and technology available for blind individuals.

Visual experience shapes the Bouba-Kiki effect and the size-weight illusion upon sight restoration from congenital blindness

The Bouba-Kiki effect is the systematic mapping between round/spiky shapes and speech sounds ("Bouba"/"Kiki"). In the size-weight illusion, participants judge the smaller of two equally-weighted objects as being heavier. Here we investigated the contribution of visual experience to the development of these phenomena. We compared three groups: early blind individuals (no visual experience), individuals treated for congenital cataracts years after birth (late visual experience), and typically sighted controls (visual experience from birth). We found that, in cataract-treated participants (tested visually/visuo-haptically), both phenomena are absent shortly after sight onset, just like in blind individuals (tested haptically). However, they emerge within months following surgery, becoming statistically indistinguishable from the sighted controls. This suggests a pivotal role of visual experience and refutes the existence of an early sensitive period: A short period of experience, even when gained only years after birth, is sufficient for participants to visually pick-up regularities in the environment, contributing to the development of these phenomena.

"Impossible" Somatosensation and the (Ir)rationality of Perception

Impossible figures represent the world in ways it cannot be. From the work of M. C. Escher to any popular perception textbook, such experiences show how some principles of mental processing can be so entrenched and inflexible as to produce absurd and even incoherent outcomes that could not occur in reality. However, impossible experiences of this sort are mostly limited to visual perception; are there “impossible figures” for other sensory modalities? Here, we import a known magic trick into the laboratory to report and investigate an impossible experience for somatosensation—one that can be physically felt. We show that, even under full-cue conditions with objects that can be freely inspected, subjects can be made to experience a single object alone as feeling heavier than a group of objects that includes the single object as a member—an impossible and phenomenologically striking experience of weight. Moreover, we suggest that this phenomenon—a special case of the size-weight illusion—reflects a kind of “anti-Bayesian” perceptual updating that amplifies a challenge to rational models of perception and cognition.

The size-weight illusion is unimpaired in individuals with a history of congenital visual deprivation

Visual deprivation in childhood can lead to lifelong impairments in multisensory processing. Here, the Size-Weight Illusion (SWI) was used to test whether visuo-haptic integration recovers after early visual deprivation. Normally sighted individuals perceive larger objects to be lighter than smaller objects of the same weight. In Experiment 1, individuals treated for dense bilateral congenital cataracts (who had no patterned visual experience at birth), individuals treated for developmental cataracts (who had patterned visual experience at birth, but were visually impaired), congenitally blind individuals and normally sighted individuals had to rate the weight of manually explored cubes that differed in size (Small, Medium, Large) across two possible weights (350 g, 700 g). In Experiment 2, individuals treated for dense bilateral congenital cataracts were compared to sighted individuals in a similar task using a string set-up, which removed haptic size cues. In both experiments, indistinguishable SWI effects were observed across all groups. These results provide evidence that early aberrant vision does not interfere with the development of the SWI, and suggest a recovery of the integration of size and weight cues provided by the visual and haptic modality.

Early visual cortex response for sound in expert blind echolocators, but not in early blind non-echolocators

Echolocation is a perceptual and navigational skill that can be acquired by some individuals. Regarding blind people, this skill can help them "see" the environment around them via a new form of auditory information based on echoes. Expert human echolocators benefit from using this technique not only in controlled environments but also in their everyday lives. In the current study, we investigate the effect of echolocation on blind people's auditory spatial abilities at the cortical level. In an auditory spatial bisection task, we tested people who are early blinds and early blind expert echolocators, along with sighted people. Our results showed that there is similar early activation (50-90 ms) in the posterior area of the scalp for both early blind expert echolocators and sighted participants, but not in the early blind group. This activation was related to sound stimulation, and it is contralateral to the position of the sound in space. These findings indicate that echolocation is a good substitute for the visual modality that enables the development of auditory spatial representations when vision is not available.

The size-weight illusion comes along with improved weight discrimination

When people judge the weight of two objects of equal mass but different size, they perceive the smaller one as being heavier. Up to date, there is no consensus about the mechanisms which give rise to this size-weight illusion. We recently suggested a model that describes heaviness perception as a weighted average of two sensory heaviness estimates with correlated noise: one estimate derived from mass, the other one derived from density. The density estimate is first derived from mass and size, but at the final perceptual level, perceived heaviness is biased by an object's density, not by its size. Here, we tested the models' prediction that weight discrimination of equal-size objects is better in lifting conditions which are prone to the size-weight illusion as compared to conditions lacking (the essentially uninformative) size information. This is predicted because in these objects density covaries with mass, and according to the model density serves as an additional sensory cue. Participants performed a two-interval forced-choice weight discrimination task. We manipulated the quality of either haptic (Experiment 1) or visual (Experiment 2) size information and measured just-noticeable differences (JNDs). Both for the haptic and the visual illusion, JNDs were lower in lifting conditions in which size information was available. Thus, when heaviness perception can be influenced by an object's density, it is more reliable. This discrimination benefit under conditions that provide the additional information that objects are of equal size is further support for the role of density and the integration of sensory estimates in the size-weight illusion.

A meta-analysis of the size-weight and material-weight illusions

The current study comprises the first systematic meta-analysis of weight illusions. We obtained descriptive data from studies in which subjective heaviness estimates were made for pairs or groups of objects that had the same mass and different volumes (size-weight illusion; SWI) or different apparent material properties (material-weight illusion; MWI). Using these data, we calculated mean effect sizes to represent illusion strength. Other study details, including stimulus mass, volume, density, and degree of visual and somatosensory access to the stimuli were also recorded to quantify the contribution of these variables to effect sizes for the SWI. The results indicate that the SWI has a larger mean effect size than the MWI and that the former is consistent in strength when information about stimulus size is gained through somatosensory channels, regardless of visual access. The SWI is weaker when only the visual system provides size information. Effect sizes for the SWI were larger when there was a greater difference in volume across the stimuli. There was also a positive correlation between SWI strength and the difference in physical density across the different experimental stimuli, even after controlling for volume differences. Together, we argue that these findings provide support for theories of weight illusions that are based on conceptual expectancies as well as those that are based on bottom-up processing of physical density. We further propose that these processes, which have been considered dichotomously in the past, may not be mutually exclusive from each other and could both contribute to our perception of weight when we handle objects in everyday life.

The flexible action system: Click-based echolocation may replace certain visual functionality for adaptive walking

People use sensory, in particular visual, information to guide actions such as walking around obstacles, grasping or reaching. However, it is presently unclear how malleable the sensorimotor system is. The present study investigated this by measuring how click-based echolocation may be used to avoid obstacles while walking. We tested 7 blind echolocation experts, 14 sighted, and 10 blind echolocation beginners. For comparison, we also tested 10 sighted participants, who used vision. To maximize the relevance of our research for people with vision impairments, we also included a condition where the long cane was used and considered obstacles at different elevations. Motion capture and sound data were acquired simultaneously. We found that echolocation experts walked just as fast as sighted participants using vision, and faster than either sighted or blind echolocation beginners. Walking paths of echolocation experts indicated early and smooth adjustments, similar to those shown by sighted people using vision and different from later and more abrupt adjustments of beginners. Further, for all participants, the use of echolocation significantly decreased collision frequency with obstacles at head, but not ground level. Further analyses showed that participants who made clicks with higher spectral frequency content walked faster, and that for experts higher clicking rates were associated with faster walking. The results highlight that people can use novel sensory information (here, echolocation) to guide actions, demonstrating the action system’s ability to adapt to changes in sensory input. They also highlight that regular use of echolocation enhances sensory-motor coordination for walking in blind people.

Vision loss has negative consequences for people’s mobility. The current report demonstrates that echolocation might replace certain visual functionality for adaptive walking. Importantly, the report also highlights that echolocation and long cane are complementary mobility techniques. The findings have direct relevance for professionals involved in mobility instruction and for people who are blind.

Container size exerts a stronger influence than liquid volume on the perceived weight of objects

Many features of an object can influence how we predict and perceive its weight. The current study evaluated the relative contributions of sensory and conceptual processing of object features on weight perception. We employed a novel paradigm to investigate how container size and the amount of liquid inside can influence the perceived weight of bottles and the forces deployed when lifting them. Stimulus pairs always had the same mass but could vary in liquid volume (full vs half-full bottle) or size (large vs small bottle; size-weight illusion (SWI)). In Experiment 1, participants lifted the stimuli via strings, which served to isolate the influence of visual from kinaesthetic information about the size of stimuli on perception and lifting behaviour. In Experiment 2, participants lifted the stimuli via handles that were attached directly to the objects. This lifting style is more likely to include deviations from true vertical lifting, which should theoretically provide more kinaesthetic information about the size of the stimuli. Experiment 1 did not produce any weight illusion. Experiment 2 produced a weight illusion but only when container size differed. Thus, liquid volume did not influence perceived weight when container size was held constant in either experiment. Curiously, additional control experiments revealed that participants could not discriminate between the different sized bottles solely from the kinaesthetic information received from a handle-based lift, suggesting that size might be processed differently when making explicit perceptual judgements about it than when influencing weight perception. Together, these findings suggest that weight illusions are driven more strongly by the kinaesthetic processing of stimulus features than predictions arising from conceptual weight cues.

Move on up: Fingertip forces and felt heaviness are modulated by the goal of the lift

When we interact with objects, we usually do so for a purpose. It is well known that the specific goal of an action can have a substantial effect on initial reach kinematics. No research, however, has examined the effect that the goal of a lift can have on the fingertip forces and perception of object weight when picking up an object to move it. Here, we report a study in which participants were asked to move objects laterally to a higher platform, to a lower platform, or to a platform of the same height. The objects were rated, on average, as feeling heavier after they were moved to a higher platform than after they were moved to a lower platform or to a platform of the same height. Furthermore, participants gripped and lifted with more force, and used higher rates of force, when moving objects to a higher platform compared with moving it to a platform of the same height. These findings suggest that the goal of movement in the context of object interaction may affect how heavy an object feels and the way in which it is lifted.

Examining the size-weight illusion with visuo-haptic conflict in immersive virtual reality

When we experience our environment, we do so by combining sensory inputs with expectations derived from our prior knowledge, which can lead to surprising perceptual effects such as small objects feeling heavier than equally weighted large objects (the size-weight illusion (SWI)). Interestingly, there is evidence that the way in which the volume of an object is experienced can affect the strength of the illusion, with a SWI induced by exclusively haptic volume cues feeling stronger than a SWI induced with only visual volume cues. Furthermore, visual cues appear to add nothing over and above haptic size cues in terms of the strength of the induced weight illusion-findings which are difficult to reconcile with work using cue-conflict paradigms where visual cues usually dominate haptic cues. Here, virtual reality was used to place these senses in conflict with one another. Participants (N = 22) judged the heaviness of identically weighted cylinders across three conditions: (1) objects appeared different sizes but were physically the same size, (2) objects were physically different sizes but appeared to be the same size, or (3) objects which looked and felt different sizes from one another. Consistent with prior work, haptic size cues induced a larger SWI than that induced by visual size differences. In contrast to prior work, however, congruent vision and haptic size cues yielded a larger still SWI. These findings not only add to our understanding of how different modalities combine to influence our hedonic perception but also showcase how virtual reality can develop novel cue-conflict paradigms.

Preserved Object Weight Processing after Bilateral Lateral Occipital Complex Lesions

Object interaction requires knowledge of the weight of an object, as well as its shape. The lateral occipital complex (LOC), an area within the ventral visual pathway, is well known to be critically involved in processing visual shape information. Recently, however, LOC has also been implicated in coding object weight before grasping-a result that is surprising because weight is a nonvisual object property that is more relevant for motor interaction than visual perception. Here, we examined the causal role of LOC in perceiving heaviness and in determining appropriate fingertip forces during object lifting. We studied perceptions of heaviness and lifting behavior in a neuropsychological patient (M.C.) who has large bilateral occipitotemporal lesions that include LOC. We compared the patient's performance to a group of 18 neurologically healthy age-matched controls. Participants were asked to lift and report the perceived heaviness of a set of equally weighted spherical objects of various sizes-stimuli which typically induce the size-weight illusion, in which the smaller objects feel heavier than the larger objects despite having identical mass. Despite her ventral stream lesions, M.C. experienced a robust size-weight illusion induced by visual cues to object volume, and the magnitude of the illusion in M.C. was comparable to age-matched controls. Similarly, M.C. evinced predictive fingertip force scaling to visual size cues during her initial lifts of the objects that were well within the normal range. These single-case neuropsychological findings suggest that LOC is unlikely to play a causal role in computing object weight.

How Prior Expectations Influence Older Adults’ Perception and Action During Object Interaction

The apparent size of an object can influence how we interact with and perceive the weight of objects in our environment. Little is known, however, about how this cue affects behaviour across the lifespan. Here, in the context of the size–weight illusion, we examined how visual size cues influenced the predictive application of fingertip forces and perceptions of heaviness in a group of older participants. We found that our older sample experienced a robust size–weight illusion, which did not differ from that experienced by younger participants. Older and young participants also experienced a real weight difference to a similar degree. By contrast, compared to younger participants our older group showed no evidence that size cues influenced the way they initially gripped and lifted the objects. These results highlight a unique dissociation between how perception and action diverge across the lifespan, and suggest that deficits in the ability to use prediction to guide actions might underpin some of the manual interaction difficulties experienced by the older adults.

Visual sensory stimulation interferes with people's ability to echolocate object size

Echolocation is the ability to use sound-echoes to infer spatial information about the environment. People can echolocate for example by making mouth clicks. Previous research suggests that echolocation in blind people activates brain areas that process light in sighted people. Research has also shown that echolocation in blind people may replace vision for calibration of external space. In the current study we investigated if echolocation may also draw on ‘visual’ resources in the sighted brain. To this end, we paired a sensory interference paradigm with an echolocation task. We found that exposure to an uninformative visual stimulus (i.e. white light) while simultaneously echolocating significantly reduced participants’ ability to accurately judge object size. In contrast, a tactile stimulus (i.e. vibration on the skin) did not lead to a significant change in performance (neither in sighted, nor blind echo expert participants). Furthermore, we found that the same visual stimulus did not affect performance in auditory control tasks that required detection of changes in sound intensity, sound frequency or sound location. The results suggest that processing of visual and echo-acoustic information draw on common neural resources.

Echolocation in humans: an overview

Bats and dolphins are known for their ability to use echolocation. They emit bursts of sounds and listen to the echoes that bounce back to detect the objects in their environment. What is not as well-known is that some blind people have learned to do the same thing, making mouth clicks, for example, and using the returning echoes from those clicks to sense obstacles and objects of interest in their surroundings. The current review explores some of the research that has examined human echolocation and the changes that have been observed in the brains of echolocation experts. We also discuss potential applications and assistive technology based on echolocation. Blind echolocation experts can sense small differences in the location of objects, differentiate between objects of various sizes and shapes, and even between objects made of different materials, just by listening to the reflected echoes from mouth clicks. It is clear that echolocation may enable some blind people to do things that are otherwise thought to be impossible without vision, potentially providing them with a high degree of independence in their daily lives and demonstrating that echolocation can serve as an effective mobility strategy in the blind. Neuroimaging has shown that the processing of echoes activates brain regions in blind echolocators that would normally support vision in the sighted brain, and that the patterns of these activations are modulated by the information carried by the echoes. This work is shedding new light on just how plastic the human brain is. WIREs Cogn Sci 2016, 7:382-393. doi: 10.1002/wcs.1408 For further resources related to this article, please visit the WIREs website.

Perceiving and acting upon weight illusions in the absence of somatosensory information

When lifting novel objects, individuals' fingertip forces are influenced by a variety of cues such as volume and apparent material. This means that heavy-looking objects tend to be lifted with more force than lighter-looking objects, even when they weigh the same amount as one another. Expectations about object weight based on visual appearance also influence how heavy an object feels when it is lifted. For instance, in the "size-weight illusion," small objects feel heavier than equally weighted large objects. Similarly, in the "material-weight illusion," objects that seem to be made from light-looking materials feel heavier than objects of the same weight that appear to be made from heavy-looking materials. In this study, we investigated these perceptual and sensorimotor effects in IW, an individual with peripheral deafferentation (i.e., a loss of tactile and proprioception feedback). We examined his perceptions of heaviness and fingertip force application over repeated lifts of objects that varied in size or material properties. Despite being able to report real weight differences, IW did not appear to experience the size- or material-weight illusions. Furthermore, he showed no evidence of sensorimotor prediction based on size and material cues. The results are discussed in the context of forward models and their possible influence on weight perception and fingertip force control.

Task-dependent calibration of auditory spatial perception through environmental visual observation

Visual information is paramount to space perception. Vision influences auditory space estimation. Many studies show that simultaneous visual and auditory cues improve precision of the final multisensory estimate. However, the amount or the temporal extent of visual information, that is sufficient to influence auditory perception, is still unknown. It is therefore interesting to know if vision can improve auditory precision through a short-term environmental observation preceding the audio task and whether this influence is task-specific or environment-specific or both. To test these issues we investigate possible improvements of acoustic precision with sighted blindfolded participants in two audio tasks [minimum audible angle (MAA) and space bisection] and two acoustically different environments (normal room and anechoic room). With respect to a baseline of auditory precision, we found an improvement of precision in the space bisection task but not in the MAA after the observation of a normal room. No improvement was found when performing the same task in an anechoic chamber. In addition, no difference was found between a condition of short environment observation and a condition of full vision during the whole experimental session. Our results suggest that even short-term environmental observation can calibrate auditory spatial performance. They also suggest that echoes can be the cue that underpins visual calibration. Echoes may mediate the transfer of information from the visual to the auditory system.