The future of sensory substitution, addition, and expansion via haptic devices

Cross-Modal Sensory Boosting to Improve High-Frequency Hearing Loss: Device Development and Validation

Background

High-frequency hearing loss is one of the most common problems in the aging population and with those who have a history of exposure to loud noises. This type of hearing loss can be frustrating and disabling, making it difficult to understand speech communication and interact effectively with the world.

Objective

This study aimed to examine the impact of spatially unique haptic vibrations representing high-frequency phonemes on the self-perceived ability to understand conversations in everyday situations.

Methods

To address high-frequency hearing loss, a multi-motor wristband was developed that uses machine learning to listen for specific high-frequency phonemes. The wristband vibrates in spatially unique locations to represent which phoneme was present in real time. A total of 16 participants with high-frequency hearing loss were recruited and asked to wear the wristband for 6 weeks. The degree of disability associated with hearing loss was measured weekly using the Abbreviated Profile of Hearing Aid Benefit (APHAB).

Results

By the end of the 6-week study, the average APHAB benefit score across all participants reached 12.39 points, from a baseline of 40.32 to a final score of 27.93 (SD 13.11; N=16; P=.002, 2-tailed dependent t test). Those without hearing aids showed a 10.78-point larger improvement in average APHAB benefit score at 6 weeks than those with hearing aids (t14=2.14; P=.10, 2-tailed independent t test). The average benefit score across all participants for ease of communication was 15.44 (SD 13.88; N=16; P<.001, 2-tailed dependent t test). The average benefit score across all participants for background noise was 10.88 (SD 17.54; N=16; P=.03, 2-tailed dependent t test). The average benefit score across all participants for reverberation was 10.84 (SD 16.95; N=16; P=.02, 2-tailed dependent t test).

Conclusions

These findings show that vibrotactile sensory substitution delivered by a wristband that produces spatially distinguishable vibrations in correspondence with high-frequency phonemes helps individuals with high-frequency hearing loss improve their perceived understanding of verbal communication. Vibrotactile feedback provides benefits whether or not a person wears hearing aids, albeit in slightly different ways. Finally, individuals with the greatest perceived difficulty understanding speech experienced the greatest amount of perceived benefit from vibrotactile feedback.

Improved speech intelligibility in the presence of congruent vibrotactile speech input

Vibrotactile stimulation is believed to enhance auditory speech perception, offering potential benefits for cochlear implant (CI) users who may utilize compensatory sensory strategies. Our study advances previous research by directly comparing tactile speech intelligibility enhancements in normal-hearing (NH) and CI participants, using the same paradigm. Moreover, we assessed tactile enhancement considering stimulus non-specific, excitatory effects through an incongruent audio-tactile control condition that did not contain any speech-relevant information. In addition to this incongruent audio-tactile condition, we presented sentences in an auditory only and a congruent audio-tactile condition, with the congruent tactile stimulus providing low-frequency envelope information via a vibrating probe on the index fingertip. The study involved 23 NH listeners and 14 CI users. In both groups, significant tactile enhancements were observed for congruent tactile stimuli (5.3% for NH and 5.4% for CI participants), but not for incongruent tactile stimulation. These findings replicate previously observed tactile enhancement effects. Juxtaposing our study with previous research, the informational content of the tactile stimulus emerges as a modulator of intelligibility: Generally, congruent stimuli enhanced, non-matching tactile stimuli reduced, and neutral stimuli did not change test outcomes. We conclude that the temporal cues provided by congruent vibrotactile stimuli may aid in parsing continuous speech signals into syllables and words, consequently leading to the observed improvements in intelligibility.

Sensory substitution increases robotic surgical performance and sets the ground for a mediating role of the sense of embodiment: a systematic review

Sensory Substitution (SS) allows the elaboration of information via non preferential sensory modalities. This phenomenon occurs in robotic-assisted surgery (RAS), in which haptic feedback is lacking. It has been suggested that SS could sustain surgeons' proficiency by means of visual clues for inferring tactile information, that also promotes the feeling of haptic phantom sensations. A critical role in reaching a good performance in procedural tasks is also sustained by the Sense of Embodiment (SE), that is, the capacity to integrate objects into subjective bodily self-representation. As SE is enhanced by haptic sensations, we hypothesize a role of SS in promoting SE in RAS. Accordingly, the goal of this systematic review is to summarize the evidence pertaining the study of SS in RAS in order to highlight the impact on the performance, and to identify a mediating role of the SE in increasing dexterity in RAS. Eight studies selected from the MEDLINE and Scopus® databases met inclusion criteria for a qualitative synthesis. Results indicated that haptic to other modalities SS enhanced force consistency and accuracy, and decreased surgeon fatigue. Expert surgeons, as compared to novices, showed a better natural SS processing, testified by a proficient performance with and without SS aids. No studies investigated the mediating role of SE. These findings indicate that SS is subjected to learning and memory processes that help surgeons to rapidly derive haptic-correlates from visual clues, which are highly required for a good performance. Also, the higher ability of doing SS and the associated perception of haptic sensations might increase multisensory integration, which might sustain performance.