User evaluation of a communication system that automatically generates captions to improve telephone communication

Benefits of Text Supplementation on Sentence Recognition and Subjective Ratings With and Without Facial Cues for Listeners With Normal Hearing

OBJECTIVES

Recognizing speech through telecommunication can be challenging in unfavorable listening conditions. Text supplementation or provision of facial cues can facilitate speech recognition under some circumstances. However, our understanding of the combined benefit of text and facial cues in telecommunication is limited. The purpose of this study was to investigate the potential benefit of text supplementation for sentence recognition scores and subjective ratings of spoken speech with and without facial cues available.

DESIGN

Twenty adult females (M = 24 years, range 21 to 29 years) with normal hearing performed a sentence recognition task and also completed a subjective rating questionnaire in 24 conditions. The conditions varied by integrity of the available facial cues (clear facial cues, slight distortion facial cues, great distortion facial cues, no facial cues), signal-to-noise ratio (quiet, +1 dB, -3 dB), and text availability (with text, without text). When present, the text was an 86 to 88% accurate transcription of the auditory signal presented at a 500 ms delay relative to the auditory signal.

RESULTS

The benefits of text supplementation were largest when facial cues were not available and when the signal-to-noise ratio was unfavorable. Although no recognition score benefit was present in quiet, recognition benefit was significant in all levels of background noise for all levels of facial cue integrity. Moreover, participant subjective ratings of text benefit were robust and present even in the absence of recognition benefit. Consistent with previous literature, facial cues were beneficial for sentence recognition scores in the most unfavorable signal-to-noise ratio, even when greatly distorted. It is interesting that, although all levels of facial cues were beneficial for recognition scores, participants rated a significant benefit only with clear facial cues.

CONCLUSIONS

The benefit of text for auditory-only and auditory-visual speech recognition is evident in recognition scores and subjective ratings; the benefit is larger and more robust for subjective ratings than for scores. Therefore, text supplementation might provide benefit that extends beyond speech recognition scores. Combined, these findings support the use of text supplementation in telecommunication, even when facial cues are concurrently present, such as during teleconferencing or watching television.

Effects of text supplementation on speech intelligibility for listeners with normal and impaired hearing: a systematic review with implications for telecommunication

OBJECTIVE

Telecommunication can be difficult in the presence of noise or hearing loss. The purpose of this study was to systematically review evidence regarding the effects of text supplementation (e.g. captions, subtitles) of auditory or auditory-visual signals on speech intelligibility for listeners with normal or impaired hearing.

DESIGN

Three databases were searched. Articles were evaluated for inclusion based on the Population Intervention Comparison Outcome framework. The Effective Public Health Practice Project instrument was used to evaluate the quality of the identified articles.

STUDY SAMPLE

After duplicates were removed, the titles and abstracts of 2019 articles were screened. Forty-six full texts were reviewed; ten met inclusion criteria.

RESULTS

The quality of all ten articles was moderate or strong. The articles demonstrated that text added to auditory (or auditory-visual) signals improved speech intelligibility and that the benefits were largest when auditory signal integrity was low, accuracy of the text was high, and the auditory signal and text were synchronous. Age and hearing loss did not affect benefits from the addition of text.

CONCLUSIONS

Although only based on ten studies, these data support the use of text as a supplement during telecommunication, such as while watching television or during telehealth appointments.

Benefits of Bilateral Hearing on the Telephone for Cochlear Implant Recipients

BACKGROUND

 Cochlear implant (CI) recipients frequently experience difficulty understanding speech over the telephone and rely on hearing assistive technology (HAT) to improve performance. Bilateral inter-processor audio streaming technology using nearfield magnetic induction is an advanced technology incorporated within a hearing aid or CI processor that can deliver telephone audio signals captured at one sound processor to the sound processor at the opposite ear. To date, limited data exist examining the efficacy of this technology in CI users to improve speech understanding on the telephone.

PURPOSE

 The primary objective of this study was to examine telephone speech recognition outcomes in bilateral CI recipients in a bilateral inter-processor audio streaming condition (DuoPhone) compared with a monaural condition (i.e., telephone listening with one sound processor) in quiet and in background noise. Outcomes in the monaural and bilateral conditions using either a telecoil or T-Mic2 technology were also assessed. The secondary aim was to examine how deactivating microphone input in the contralateral processor in the bilateral wireless streaming conditions, and thereby modifying the signal-to-noise ratio, affected speech recognition in noise.

RESEARCH DESIGN

 A repeated-measures design was used to evaluate speech recognition performance in quiet and competing noise with the telephone signal transmitted acoustically or via the telecoil to the ipsilateral sound processor microphone in monaural and bilateral wireless streaming listening conditions.

STUDY SAMPLE

 Nine bilateral CI users with Advanced Bionics HiRes 90K and/or CII devices were included in the study.

DATA COLLECTION AND ANALYSIS

 The effects of phone input (monaural [DuoPhone Off] vs. bilateral [DuoPhone on]) and processor input (T-Mic2 vs. telecoil) on word recognition in quiet and noise were assessed using separate repeated-measures analysis of variance. Effect of the contralateral device mic deactivation on speech recognition outcomes for the T-Mic2 DuoPhone conditions was assessed using paired Student's t-tests.

RESULTS

 Telephone speech recognition was significantly better in the bilateral inter-processor streaming conditions relative to the monaural conditions in both quiet and noise. Speech recognition outcomes were similar in quiet and noise when using the T-Mic2 and telecoil in the monaural and bilateral conditions. For the acoustic DuoPhone conditions using the T-Mic2, speech recognition in noise was significantly better when the microphone of the contralateral processor was disabled.

CONCLUSION

 Inter-processor audio streaming allows for bilateral listening on the telephone and produces better speech recognition in quiet and in noise compared with monaural listening conditions for adult CI recipients.

Effects of Second Language Proficiency and Linguistic Uncertainty on Recognition of Speech in Native and Nonnative Competing Speech

PURPOSE

The purpose of this study was to investigate the effects of 2nd language proficiency and linguistic uncertainty on performance and listening effort in mixed language contexts.

METHOD

Thirteen native speakers of Dutch with varying degrees of fluency in English listened to and repeated sentences produced in both Dutch and English and presented in the presence of single-talker competing speech in both Dutch and English. Target and masker language combinations were presented in both blocked and mixed (unpredictable) conditions. In the blocked condition, in each block of trials the target-masker language combination remained constant, and the listeners were informed of both prior to beginning the block. In the mixed condition, target and masker language varied randomly from trial to trial. All listeners participated in all conditions. Performance was assessed in terms of speech reception thresholds, whereas listening effort was quantified in terms of pupil dilation.

RESULTS

Performance (speech reception thresholds) and listening effort (pupil dilation) were both affected by 2nd language proficiency (English test score) and target and masker language: Performance was better in blocked as compared to mixed conditions, with Dutch as compared to English targets, and with English as compared to Dutch maskers. English proficiency was correlated with listening performance. Listeners also exhibited greater peak pupil dilation in mixed as compared to blocked conditions for trials with Dutch maskers, whereas pupil dilation during preparation for speaking was higher for English targets as compared to Dutch ones in almost all conditions.

CONCLUSIONS

Both listener's proficiency in a 2nd language and uncertainty about the target language on a given trial play a significant role in how bilingual listeners attend to speech in the presence of competing speech in different languages, but precise effects also depend on which language is serving as target and which as masker.

Self-Assessed Hearing Handicap in Older Adults With Poorer-Than-Predicted Speech Recognition in Noise

PURPOSE

Even older adults with relatively mild hearing loss report hearing handicap, suggesting that hearing handicap is not completely explained by reduced speech audibility.

METHOD

We examined the extent to which self-assessed ratings of hearing handicap using the Hearing Handicap Inventory for the Elderly (HHIE; Ventry & Weinstein, 1982) were significantly associated with measures of speech recognition in noise that controlled for differences in speech audibility.

RESULTS

One hundred sixty-two middle-aged and older adults had HHIE total scores that were significantly associated with audibility-adjusted measures of speech recognition for low-context but not high-context sentences. These findings were driven by HHIE items involving negative feelings related to communication difficulties that also captured variance in subjective ratings of effort and frustration that predicted speech recognition. The average pure-tone threshold accounted for some of the variance in the association between the HHIE and audibility-adjusted speech recognition, suggesting an effect of central and peripheral auditory system decline related to elevated thresholds.

CONCLUSION

The accumulation of difficult listening experiences appears to produce a self-assessment of hearing handicap resulting from (a) reduced audibility of stimuli, (b) declines in the central and peripheral auditory system function, and (c) additional individual variation in central nervous system function.

Effects of noise, reverberation and foreign accent on native and non-native listeners' performance of English speech comprehension

A large number of non-native English speakers may be found in American classrooms, both as listeners and talkers. Little is known about how this population comprehends speech in realistic adverse acoustical conditions. A study was conducted to investigate the effects of background noise level (BNL), reverberation time (RT), and talker foreign accent on native and non-native listeners' speech comprehension, while controlling for English language abilities. A total of 115 adult listeners completed comprehension tasks under 15 acoustic conditions: three BNLs (RC-30, RC-40, and RC-50) and five RTs (from 0.4 to 1.2 s). Fifty-six listeners were tested with speech from native English-speaking talkers and 59 with native Mandarin-Chinese-speaking talkers. Results show that, while higher BNLs were generally more detrimental to listeners with lower English proficiency, all listeners experienced significant comprehension deficits above RC-40 with native English talkers. This limit was lower (i.e., above RC-30), however, with Chinese talkers. For reverberation, non-native listeners as a group performed best with RT up to 0.6 s, while native listeners performed equally well up to 1.2 s. A matched foreign accent benefit has also been identified, where the negative impact of higher reverberation does not exist for non-native listeners who share the talker's native language.

Spatial separation benefit for unaided and aided listening

OBJECTIVES

The benefit attributable to spatial separation of speech and noise was measured as a function of low-pass cutoff frequency with and without bilateral hearing aids.

DESIGN

Fourteen younger and 10 older adults with normal hearing and 12 older adults with mild-to-moderate sloping high frequency hearing loss were included to assess the effects of age and hearing loss. Subjects with hearing loss were provided commercially available bilateral hearing aids. Consonant recognition was measured at 70 dB SPL in a background of speech-shaped noise at 66 dB SPL. Speech and noise were low-pass filtered at 1.7, 3.4, and 7.1 kHz. Nonsense syllables were always at 0° and noise was at either 0° or 90°. Speech and noise spectra for all conditions were digitally recorded using a probe microphone placed in each ear canal of each subject. Spectra and levels of speech, and quiet thresholds for narrowband noises, were used to calculate the Articulation Index and provide predictions of unaided and aided (hearing-impaired only) consonant recognition, spatial benefit, and hearing aid benefit for each condition. Subjective ratings of workload (NASA Task Load Index) were obtained for all unaided and aided measures of speech recognition.

RESULTS

Consonant recognition in noise improved for all groups with speech and noise spatially separated and with the addition of high-frequency speech information. Scores were poorer overall for the older adults with hearing loss than for the other groups. For normal-hearing subjects, observed scores and spatial benefit were better than predicted. For hearing-impaired subjects, scores did not significantly improve with hearing aids, even with higher frequencies and spatial separation, and were poorer than predicted especially for aided listening. Similar to subjects with normal hearing, spatial benefit for hearing-impaired subjects was larger than predicted.

CONCLUSIONS

Younger and older adults with normal hearing benefited from spatial separation of speech and noise sources to a greater extent than predicted based on simple audibility. Thus, no age-related deficits in the use of interaural difference cues were observed. Although hearing aid benefit was negligible, perceived listening effort was lower aided than unaided, especially with spatial separation. Articulation Index predictions revealed that speech audibility was generally restored with hearing aids across a wide bandwidth of speech, especially in the far ear. Thus, reduced audibility was not a primary factor in limited hearing aid benefit, suggesting that peripheral, central-auditory and/or cognitive changes may have played a role. In contrast, unaided and aided spatial benefit was better than predicted, and spatial benefit was slightly larger with hearing aids than without. Thus, these older adults with hearing loss using bilateral hearing aids were able to take advantage of binaural cues to improve consonant recognition in noise.

Cognitive abilities relate to self-reported hearing disability

PURPOSE

In this explorative study, the authors investigated the relationship between auditory and cognitive abilities and self-reported hearing disability.

METHOD

Thirty-two adults with mild to moderate hearing loss completed the Amsterdam Inventory for Auditory Disability and Handicap (AIADH; Kramer, Kapteyn, Festen, & Tobi, 1996) and performed the Text Reception Threshold (TRT; Zekveld, George, Kramer, Goverts, & Houtgast, 2007) test as well as tests of spatial working memory (SWM) and visual sustained attention. Regression analyses examined the predictive value of age, hearing thresholds (pure-tone averages [PTAs]), speech perception in noise (speech reception thresholds in noise [SRTNs]), and the cognitive tests for the 5 AIADH factors.

RESULTS

Besides the variance explained by age, PTA, and SRTN, cognitive abilities were related to each hearing factor. The reported difficulties with sound detection and speech perception in quiet were less severe for participants with higher age, lower PTAs, and better TRTs. Fewer sound localization and speech perception in noise problems were reported by participants with better SRTNs and smaller SWM. Fewer sound discrimination difficulties were reported by subjects with better SRTNs and TRTs and smaller SWM.

CONCLUSIONS

The results suggest a general role of the ability to read partly masked text in subjective hearing. Large working memory was associated with more reported hearing difficulties. This study shows that besides auditory variables and age, cognitive abilities are related to self-reported hearing disability.

Measuring cognitive factors in speech comprehension: the value of using the Text Reception Threshold test as a visual equivalent of the SRT test

The ability to comprehend speech in noise is influenced by bottom-up auditory and top-down cognitive capacities. Separate examination of these capacities is relevant for various purposes. Speech-Reception-Threshold (SRT) tests measure an individual's ability to comprehend speech. This paper addresses the value of the Text-Reception-Threshold (TRT) test (a visual parallel of the SRT test) to assess the cognitive capacities allocated during speech comprehension. We conducted a secondary data analysis, including 87 normally-hearing adults (aged 18 to 78 years). Correlation coefficients between age, TRT, working memory (Spatial Span) and SRT were examined. The TRT and SRT correlated significantly (r = 0.30), supporting the value of TRT in explaining inter-individual differences in SRTs. The relations between age and TRT and between SSP and TRT were non-significant. The results indicate that the current TRT test does not fully cover the cognitive aspects relevant in speech comprehension. Adaptation of the test is required before clinical implementation can be considered.