Human Click-Based Echolocation of Distance: Superfine Acuity and Dynamic Clicking Behaviour

Preliminary investigation of SEZUAL device for basic material identification and simple spatial navigation for blind and visually impaired people

PURPOSE

we present a preliminary set of experimental studies that demonstrates device-aided echolocation enabling in blind and visually impaired individuals. The proposed device emits a click-like sound into the surrounding space and returning sound is perceived by participants to infer the surrounding environment.

MATERIALS AND METHODS

two sets of experiments were set up to evaluate the echolocation abilities of nine blind participants. The first setup was designed to identify four material types based on the sound reflection properties of materials, such as glass, metal, wood, and ceramics. The second setup was navigation through a basic maze with the device.

RESULTS

experimental data demonstrate that the use of the proposed device enables active echolocation abilities in blind participants, particularly for material identification and spatial mobility.

CONCLUSION

the proposed device can potentially be used to rehabilitate disabled blind and visually impaired individuals in terms of spatial mobility and orientation.

Effects of type of emission and masking sound, and their spatial correspondence, on blind and sighted people's ability to echolocate

Ambient sound can mask acoustic signals. The current study addressed how echolocation in people is affected by masking sound, and the role played by type of sound and spatial (i.e. binaural) similarity. We also investigated the role played by blindness and long-term experience with echolocation, by testing echolocation experts, as well as blind and sighted people new to echolocation. Results were obtained in two echolocation tasks where participants listened to binaural recordings of echolocation and masking sounds, and either localized echoes in azimuth or discriminated echo audibility. Echolocation and masking sounds could be either clicks or broad band noise. An adaptive staircase method was used to adjust signal-to-noise ratios (SNRs) based on participants' responses. When target and masker had the same binaural cues (i.e. both were monoaural sounds), people performed better (i.e. had lower SNRs) when target and masker used different types of sound (e.g. clicks in noise-masker or noise in clicks-masker), as compared to when target and masker used the same type of sound (e.g. clicks in click-, or noise in noise-masker). A very different pattern of results was observed when masker and target differed in their binaural cues, in which case people always performed better when clicks were the masker, regardless of type of emission used. Further, direct comparison between conditions with and without binaural difference revealed binaural release from masking only when clicks were used as emissions and masker, but not otherwise (i.e. when noise was used as masker or emission). This suggests that echolocation with clicks or noise may differ in their sensitivity to binaural cues. We observed the same pattern of results for echolocation experts, and blind and sighted people new to echolocation, suggesting a limited role played by long-term experience or blindness. In addition to generating novel predictions for future work, the findings also inform instruction in echolocation for people who are blind or sighted.

Adaptive echolocation behavior of bats and toothed whales in dynamic soundscapes

Journal of Experimental Biology has a long history of reporting research discoveries on animal echolocation, the subject of this Centenary Review. Echolocating animals emit intense sound pulses and process echoes to localize objects in dynamic soundscapes. More than 1100 species of bats and 70 species of toothed whales rely on echolocation to operate in aerial and aquatic environments, respectively. The need to mitigate acoustic clutter and ambient noise is common to both aerial and aquatic echolocating animals, resulting in convergence of many echolocation features, such as directional sound emission and hearing, and decreased pulse intervals and sound intensity during target approach. The physics of sound transmission in air and underwater constrains the production, detection and localization of sonar signals, resulting in differences in response times to initiate prey interception by aerial and aquatic echolocating animals. Anti-predator behavioral responses of prey pursued by echolocating animals affect behavioral foraging strategies in air and underwater. For example, many insect prey can detect and react to bat echolocation sounds, whereas most fish and squid are unresponsive to toothed whale signals, but can instead sense water movements generated by an approaching predator. These differences have implications for how bats and toothed whales hunt using echolocation. Here, we consider the behaviors used by echolocating mammals to (1) track and intercept moving prey equipped with predator detectors, (2) interrogate dynamic sonar scenes and (3) exploit visual and passive acoustic stimuli. Similarities and differences in animal sonar behaviors underwater and in air point to open research questions that are ripe for exploration.

Auditory orientation and distance estimation of sighted humans using virtual echolocation with artificial and self-generated sounds

Active echolocation of sighted humans using predefined synthetic and self-emitted sounds, as habitually used by blind individuals, was investigated. Using virtual acoustics, distance estimation and directional localization of a wall in different rooms were assessed. A virtual source was attached to either the head or hand with realistic or increased source directivity. A control condition was tested with a virtual sound source located at the wall. Untrained echolocation performance comparable to performance in the control condition was achieved on an individual level. On average, the echolocation performance was considerably lower than in the control condition, however, it benefitted from increased directivity.

EchoRead Programme: Learning echolocation skills through self-paced professional development during the COVID-19 pandemic

Echolocation is used by people with low vision or blindness to support their navigation. Internationally, Orientation and Mobility (O&M) Specialists have learned echolocation skills and how to teach them to clients, through formal workshops with a subject matter expert. However, COVID-19 has limited access to these in-person professional development opportunities. This study investigated whether an O&M professional could learn echolocation skills in a self-paced programme with only the support of a lay assistant. We developed the EchoRead Programme to equip an individual O&M Specialist to learn basic echolocation skills in 4 hours. This auto-ethnographical perspective describes how the draft programme was trialled by one trainee O&M Specialist in her home and local neighbourhood. She developed sufficient skills to complete most of the seated, standing, and walking tasks in the programme, but needed more support developing tongue-clicking and recognising driveways when shorelining fences. She found it was important to use learning environments that were graduated in physical and audio complexity. The EchoRead Programme was then trialled and revised by an experienced O&M Specialist, beginning at home, then exploring a range of venues available within a 5 km radius – the roaming range allowed during COVID lockdown. The resulting EchoRead Programme can equip O&M professionals to be self-directed in learning early echolocation skills, using online and locally available resources. This programme could be especially useful for vision professionals and their clients, who have limited access to in-person learning opportunities with colleagues or peers because of geographical isolation, low resources, or a global pandemic.

Human Echolocators Have Better Localization Off Axis

Here, we report novel empirical results from a psychophysical experiment in which we tested the echolocation abilities of nine blind adult human experts in click-based echolocation. We found that they had better acuity in localizing a target and used lower intensity emissions (i.e., mouth clicks) when a target was placed 45° off to the side compared with when it was placed at 0° (straight ahead). We provide a possible explanation of the behavioral result in terms of binaural-intensity signals, which appear to change more rapidly around 45°. The finding that echolocators have better echo-localization off axis is surprising, because for human source localization (i.e., regular spatial hearing), it is well known that performance is best when targets are straight ahead (0°) and decreases as targets move farther to the side. This may suggest that human echolocation and source hearing rely on different acoustic cues and that human spatial hearing has more facets than previously thought.

Smart Cane Developed with DFSS, QFD, and SDE for the Visually Impaired

This study shows an application of the Design for Six Sigma (DFSS) Methodology in the field of medical engineering. This research aims to demonstrate the application of a systematic design approach in the development of the “Ocane”, an innovative concept of smart cane for visually impaired patients which was thought of in answer to the end user’s needs, deploying an easy to transport, locate, and adjust element with ultrasonic sensors and tactile feedback. DFSS is an analytical design methodology meant to organize project workflow in a sequence of specific steps. Other standardized design procedures such as Quality Function Deployment (QFD) and Stylistic Design Engineering (SDE) have been used to support DFSS in terms of targeting customer requirements and focusing on aesthetics for ergonomics analysis, respectively. First, the QFD process is introduced and applied to gather the final customer needs, completing the analysis with benchmarking and similar-thought products on the market. Afterwards, a description of the DFSS methodology and application to the case study was deployed. Thereafter, the SDE procedure is exposed by identifying the “Ocane” concept and development, and moving towards the completion of an inventive product with a creative design and careful attention to visually impaired clients’ requirements.

Effectiveness of time-varying echo information for target geometry identification in bat-inspired human echolocation

Bats use echolocation through flexible active sensing via ultrasounds to identify environments suitable for their habitat and foraging. Mimicking the sensing strategies of bats for echolocation, this study examined how humans acquire new acoustic-sensing abilities, and proposes effective strategies for humans. A target geometry identification experiment-involving 15 sighted people without experience of echolocation-was conducted using two targets with different geometries, based on a new sensing system. Broadband frequency-modulated pulses with short inter-pulse intervals (16 ms) were used as a synthetic echolocation signal. Such pulses mimic buzz signals emitted by bats for echolocation prior to capturing their prey. The study participants emitted the signal from a loudspeaker by tapping on Android devices. Because the signal included high-frequency signals up to 41 kHz, the emitted signal and echoes from a stationary or rotating target were recorded using a 1/7-scaled miniature dummy head. Binaural sounds, whose pitch was down-converted, were presented through headphones. This way, time-varying echo information was made available as an acoustic cue for target geometry identification under a rotating condition, as opposed to a stationary one. In both trials, with (i.e., training trials) and without (i.e., test trials) answer feedback immediately after the participants answered, the participants identified the geometries under the rotating condition. Majority of the participants reported using time-varying patterns in terms of echo intensity, timbre, and/or pitch under the rotating condition. The results suggest that using time-varying patterns in echo intensity, timbre, and/or pitch enables humans to identify target geometries. However, performance significantly differed by condition (i.e., stationary vs. rotating) only in the test trials. This difference suggests that time-varying echo information is effective for identifying target geometry through human echolocation especially when echolocators are unable to obtain answer feedback during sensing.

Increased emission intensity can compensate for the presence of noise in human click-based echolocation

Echolocating bats adapt their emissions to succeed in noisy environments. In the present study we investigated if echolocating humans can detect a sound-reflecting surface in the presence of noise and if intensity of echolocation emissions (i.e. clicks) changes in a systematic pattern. We tested people who were blind and had experience in echolocation, as well as blind and sighted people who had no experience in echolocation prior to the study. We used an echo-detection paradigm where participants listened to binaural recordings of echolocation sounds (i.e. they did not make their own click emissions), and where intensity of emissions and echoes changed adaptively based on participant performance (intensity of echoes was yoked to intensity of emissions). We found that emission intensity had to systematically increase to compensate for weaker echoes relative to background noise. In fact, emission intensity increased so that spectral power of echoes exceeded spectral power of noise by 12 dB in 4-kHz and 5-kHz frequency bands. The effects were the same across all participant groups, suggesting that this effect occurs independently of long-time experience with echolocation. Our findings demonstrate for the first time that people can echolocate in the presence of noise and suggest that one potential strategy to deal with noise is to increase emission intensity to maintain signal-to-noise ratio of certain spectral components of the echoes.

Perceptual constancy with a novel sensory skill

Making sense of the world requires perceptual constancy—the stable perception of an object across changes in one’s sensation of it. To investigate whether constancy is intrinsic to perception, we tested whether humans can learn a form of constancy that is unique to a novel sensory skill (here, the perception of objects through click-based echolocation). Participants judged whether two echoes were different either because: (a) the clicks were different, or (b) the objects were different. For differences carried through spectral changes (but not level changes), blind expert echolocators spontaneously showed a high constancy ability (mean d′ = 1.91) compared to sighted and blind people new to echolocation (mean d′ = 0.69). Crucially, sighted controls improved rapidly in this ability through training, suggesting that constancy emerges in a domain with which the perceiver has no prior experience. This provides strong evidence that constancy is intrinsic to human perception.

This study shows that people who learn a new skill to sense their environment - here: listening to sound echoes - can correctly represent the physical properties of objects. This result has implications for effectively rehabilitating people with sensory loss.

Stimulus uncertainty affects perception in human echolocation: Timing, level, and spectrum

The human brain may use recent sensory experience to create sensory templates that are then compared to incoming sensory input, that is, "knowing what to listen for." This can lead to greater perceptual sensitivity, as long as the relevant properties of the target stimulus can be reliably estimated from past sensory experiences. Echolocation is an auditory skill probably best understood in bats, but humans can also echolocate. Here we investigated for the first time whether echolocation in humans involves the use of sensory templates derived from recent sensory experiences. Our results showed that when there was certainty in the acoustic properties of the echo relative to the emission, either in temporal onset, spectral content or level, people detected the echo more accurately than when there was uncertainty. In addition, we found that people were more accurate when the emission's spectral content was certain but, surprisingly, not when either its level or temporal onset was certain. Importantly, the lack of an effect of temporal onset of the emission is counter to that found previously for tasks using nonecholocation sounds, suggesting that the underlying mechanisms might be different for echolocation and nonecholocation sounds. Importantly, the effects of stimulus certainty were no different for people with and without experience in echolocation, suggesting that stimulus-specific sensory templates can be used in a skill that people have never used before. From an applied perspective our results suggest that echolocation instruction should encourage users to make clicks that are similar to one another in their spectral content. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Navigation and perception of spatial layout in virtual echo-acoustic space

Successful navigation involves finding the way, planning routes, and avoiding collisions. Whilst previous research has shown that people can navigate using non-visual cues, it is not clear to what degree learned non-visual navigational abilities generalise to 'new' environments. Furthermore, the ability to successfully avoid collisions has not been investigated separately from the ability to perceive spatial layout or to orient oneself in space. Here, we address these important questions using a virtual echolocation paradigm in sighted people. Fourteen sighted blindfolded participants completed 20 virtual navigation training sessions over the course of 10 weeks. In separate sessions, before and after training, we also tested their ability to perceive the spatial layout of virtual echo-acoustic space. Furthermore, three blind echolocation experts completed the tasks without training, thus validating our virtual echo-acoustic paradigm. We found that over the course of 10 weeks sighted people became better at navigating, i.e. they reduced collisions and time needed to complete the route, and increased success rates. This also generalised to 'new' (i.e. untrained) virtual spaces. In addition, after training, their ability to judge spatial layout was better than before training. The data suggest that participants acquired a 'true' sensory driven navigational ability using echo-acoustics. In addition, we show that people not only developed navigational skills related to avoidance of collisions and finding safe passage, but also processes related to spatial perception and orienting. In sum, our results provide strong support for the idea that navigation is a skill which people can achieve via various modalities, here: echolocation.

Distance Estimation by Blindfolded Sighted Participants Using Echolocation

Auditory perceived distance can be distorted in one’s internal representation. Thus, the present study examined whether blindfolded sighted participants could reduce the bias and preserve estimated distance for 5 to 15 s using echolocation. The participants performed a delayed reproduction task that consisted of testing sessions on 2 separate days in which the target distance was manipulated from 20 to 50 cm. Participants were blindfolded and asked to reproduce the distance of a target after a temporal delay of several seconds using click bursts produced by a loudspeaker. The testing session was preceded by a practice session that included training and feedback. The relationship between estimated and actual distances was approximated based on a power function and the over- and underestimation of the target distance on each test day. Although participants showed systematic bias in distance estimation on both days, participants changed their bias in the second session by shifting reproduced locations closer to their bodies. The accuracy and consistency of their responses improved across the 2 days. Neither accuracy nor consistency was affected by the retention intervals. These enhancements of performance might be due to improved hearing ability or calibration of internal spatial references through a practice session.

The flexible action system: Click-based echolocation may replace certain visual functionality for adaptive walking

People use sensory, in particular visual, information to guide actions such as walking around obstacles, grasping or reaching. However, it is presently unclear how malleable the sensorimotor system is. The present study investigated this by measuring how click-based echolocation may be used to avoid obstacles while walking. We tested 7 blind echolocation experts, 14 sighted, and 10 blind echolocation beginners. For comparison, we also tested 10 sighted participants, who used vision. To maximize the relevance of our research for people with vision impairments, we also included a condition where the long cane was used and considered obstacles at different elevations. Motion capture and sound data were acquired simultaneously. We found that echolocation experts walked just as fast as sighted participants using vision, and faster than either sighted or blind echolocation beginners. Walking paths of echolocation experts indicated early and smooth adjustments, similar to those shown by sighted people using vision and different from later and more abrupt adjustments of beginners. Further, for all participants, the use of echolocation significantly decreased collision frequency with obstacles at head, but not ground level. Further analyses showed that participants who made clicks with higher spectral frequency content walked faster, and that for experts higher clicking rates were associated with faster walking. The results highlight that people can use novel sensory information (here, echolocation) to guide actions, demonstrating the action system’s ability to adapt to changes in sensory input. They also highlight that regular use of echolocation enhances sensory-motor coordination for walking in blind people.

Vision loss has negative consequences for people’s mobility. The current report demonstrates that echolocation might replace certain visual functionality for adaptive walking. Importantly, the report also highlights that echolocation and long cane are complementary mobility techniques. The findings have direct relevance for professionals involved in mobility instruction and for people who are blind.