Time-compressed visual speech and age: a first report

Age Differences in the Effects of Speaking Rate on Auditory, Visual, and Auditory-Visual Speech Perception

OBJECTIVES

This study was designed to examine how speaking rate affects auditory-only, visual-only, and auditory-visual speech perception across the adult lifespan. In addition, the study examined the extent to which unimodal (auditory-only and visual-only) performance predicts auditory-visual performance across a range of speaking rates. The authors hypothesized significant Age × Rate interactions in all three modalities and that unimodal performance would account for a majority of the variance in auditory-visual speech perception for speaking rates that are both slower and faster than normal.

DESIGN

Participants (N = 145), ranging in age from 22 to 92, were tested in conditions with auditory-only, visual-only, and auditory-visual presentations using a closed-set speech perception test. Five different speaking rates were presented in each modality: an unmodified (normal rate), two rates that were slower than normal, and two rates that were faster than normal. Signal to noise ratios were set individually to produce approximately 30% correct identification in the auditory-only condition and this signal to noise ratio was used in the auditory-only and auditory-visual conditions.

RESULTS

Age × Rate interactions were observed for the fastest speaking rates in both the visual-only and auditory-visual conditions. Unimodal performance accounted for at least 60% of the variance in auditory-visual performance for all five speaking rates.

CONCLUSIONS

The findings demonstrate that the disproportionate difficulty that older adults have with rapid speech for auditory-only presentations can also be observed with visual-only and auditory-visual presentations. Taken together, the present analyses of age and individual differences indicate a generalized age-related decline in the ability to understand speech produced at fast speaking rates. The finding that auditory-visual speech performance was almost entirely predicted by unimodal performance across all five speaking rates has important clinical implications for auditory-visual speech perception and the ability of older adults to use visual speech information to compensate for age-related hearing loss.

Audiovisual speech perception: A new approach and implications for clinical populations

This selected overview of audiovisual (AV) speech perception examines the influence of visible articulatory information on what is heard. Thought to be a cross-cultural phenomenon that emerges early in typical language development, variables that influence AV speech perception include properties of the visual and the auditory signal, attentional demands, and individual differences. A brief review of the existing neurobiological evidence on how visual information influences heard speech indicates potential loci, timing, and facilitatory effects of AV over auditory only speech. The current literature on AV speech in certain clinical populations (individuals with an autism spectrum disorder, developmental language disorder, or hearing loss) reveals differences in processing that may inform interventions. Finally, a new method of assessing AV speech that does not require obvious cross-category mismatch or auditory noise was presented as a novel approach for investigators.

Age-related differences in inhibitory control predict audiovisual speech perception

Audiovisual (AV) speech perception is the process by which auditory and visual sensory signals are integrated and used to understand what a talker is saying during face-to-face communication. This form of communication is markedly superior to speech perception in either sensory modality alone. However, there are additional lexical factors that are affected by age-related cognitive changes that may contribute to differences in AV perception. In the current study, we extended an existing model of spoken word identification to the AV domain, and examined the cognitive factors that contribute to age-related and individual differences in AV perception of words varying in lexical difficulty (i.e., on the basis of competing items). Young (n = 49) and older adults (n = 50) completed a series of cognitive inhibition tasks and a spoken word identification task. The words were presented in auditory-only, visual-only, and AV conditions, and were equally divided into lexically hard (words with many competitors) and lexically easy (words with few competitors). Overall, young adults demonstrated better inhibitory abilities and higher identification performance than older adults. However, whereas no relationship was observed between inhibitory abilities and AV word identification performance in young adults, there was a significant relationship between Stroop interference and AV identification of lexically hard words in older adults. These results are interpreted within the framework of existing models of spoken-word recognition with implications for how cognitive deficits in older adults contribute to speech perception.

Some behavioral and neurobiological constraints on theories of audiovisual speech integration: a review and suggestions for new directions

Summerfield (1987) proposed several accounts of audiovisual speech perception, a field of research that has burgeoned in recent years. The proposed accounts included the integration of discrete phonetic features, vectors describing the values of independent acoustical and optical parameters, the filter function of the vocal tract, and articulatory dynamics of the vocal tract. The latter two accounts assume that the representations of audiovisual speech perception are based on abstract gestures, while the former two assume that the representations consist of symbolic or featural information obtained from visual and auditory modalities. Recent converging evidence from several different disciplines reveals that the general framework of Summerfield's feature-based theories should be expanded. An updated framework building upon the feature-based theories is presented. We propose a processing model arguing that auditory and visual brain circuits provide facilitatory information when the inputs are correctly timed, and that auditory and visual speech representations do not necessarily undergo translation into a common code during information processing. Future research on multisensory processing in speech perception should investigate the connections between auditory and visual brain regions, and utilize dynamic modeling tools to further understand the timing and information processing mechanisms involved in audiovisual speech integration.

Recognition of rapid speech by blind and sighted older adults

PURPOSE

To determine whether older blind participants recognize time-compressed speech better than older sighted participants.

METHOD

Three groups of adults with normal hearing participated (n = 10/group): (a) older sighted, (b) older blind, and (c) younger sighted listeners. Low-predictability sentences that were uncompressed (0% time compression ratio [TCR]) and compressed at 3 rates (40%, 50%, and 60% TCR) were presented to listeners in quiet and noise.

RESULTS

Older blind listeners recognized all time-compressed speech stimuli significantly better than did older sighted listeners in quiet. In noise, the older blind adults recognized the uncompressed and 40% TCR speech stimuli better than did the older sighted adults. Performance differences between the younger sighted adults and older blind adults were not observed.

CONCLUSIONS

The findings support the notion that older blind adults recognize time-compressed speech considerably better than older sighted adults in quiet and noise. Their performance levels are similar to those of younger adults, suggesting that age-related difficulty in understanding time-compressed speech is not an inevitable consequence of aging. Instead, frequent listening to speech at rapid rates, which was highly correlated with performance of the older blind adults, may be a useful technique to minimize age-related slowing in speech understanding.

Assessing multimodal spoken word-in-sentence recognition in children with normal hearing and children with cochlear implants

PURPOSE

To examine multimodal spoken word-in-sentence recognition in children.

METHOD

Two experiments were undertaken. In Experiment 1, the youngest age with which the multimodal sentence recognition materials could be used was evaluated. In Experiment 2, lexical difficulty and presentation modality effects were examined, along with test-retest reliability and validity in normal-hearing children and those with cochlear implants.

RESULTS

Normal-hearing children as young as 3.25 years and those with cochlear implants just under 4 years who have used their device for at least 1 year were able to complete the multimodal sentence testing. Both groups identified lexically easy words in sentences more accurately than lexically hard words across modalities, although the largest effects occurred in the auditory-only modality. Both groups displayed audiovisual integration with the highest scores achieved in the audiovisual modality, followed sequentially by auditory-only and visual-only modalities. Recognition of words in sentences was correlated with recognition of words in isolation. Preliminary results suggest fair-to-good test-retest reliability.

CONCLUSIONS

The results suggest that children's audiovisual word-in-sentence recognition can be assessed using the materials developed for this investigation. With further development, the materials hold promise for becoming a test of multimodal sentence recognition for children with hearing loss.

Lipreading, processing speed, and working memory in younger and older adults

PURPOSE

To examine several cognitive and perceptual abilities--including working memory (WM), information processing speed (PS), perceptual closure, and perceptual disembedding skill--as factors contributing to individual differences in lipreading performance and to examine how patterns in predictor variables change across age groups.

METHOD

Forty-three younger adults (mean age = 20.8 years, SD = 2.4) and 38 older adults (mean age = 76.8 years, SD = 5.6) completed tasks measuring lipreading ability, verbal WM, spatial WM (SWM), PS, and perceptual abilities.

RESULTS

Younger adults demonstrated superior lipreading ability and perceptual skills compared with older adults. In addition, younger participants exhibited longer WM spans and faster PS than did the older participants. SWM and PS accounted for a significant proportion of the variance in lipreading ability in both younger and older adults, and the pattern of predictor variables remained consistent over age groups.

CONCLUSIONS

These findings suggest that the large individual variability in lipreading ability can be explained, in part, by individual differences in SWM and PS. Furthermore, as both of these abilities are known to decline with age, the findings suggest that age-related impairments in either or both of these abilities may account for the poorer lipreading ability of older compared with younger adults.

Processing of fast speech by elderly listeners

This study investigates the relative contributions of auditory and cognitive factors to the common finding that an increase in speech rate affects elderly listeners more than young listeners. Since a direct relation between non-auditory factors, such as age-related cognitive slowing, and fast speech performance has been difficult to demonstrate, the present study took an on-line, rather than off-line, approach and focused on processing time. Elderly and young listeners were presented with speech at two rates of time compression and were asked to detect pre-assigned target words as quickly as possible. A number of auditory and cognitive measures were entered in a statistical model as predictors of elderly participants' fast speech performance: hearing acuity, an information processing rate measure, and two measures of reading speed. The results showed that hearing loss played a primary role in explaining elderly listeners' increased difficulty with fast speech. However, non-auditory factors such as reading speed and the extent to which participants were affected by increased rate of presentation in a visual analog of the listening experiment also predicted fast speech performance differences among the elderly participants. These on-line results confirm that slowed information processing is indeed part of elderly listeners' problem keeping up with fast language.

Auditory and visual lexical neighborhoods in audiovisual speech perception

Much evidence suggests that the mental lexicon is organized into auditory neighborhoods, with words that are phonologically similar belonging to the same neighborhood. In this investigation, we considered the existence of visual neighborhoods. When a receiver watches someone speak a word, a neighborhood of homophenes (ie, words that look alike on the face, such as pat and bat) is activated. The simultaneous activation of a word's auditory and visual neighborhoods may, in part, account for why individuals recognize speech better in an auditory-visual condition than what would be predicted by their performance in audition-only and vision-only conditions. A word test was administered to 3 groups of participants in audition-only, vision-only, and auditory-visual conditions, in the presence of 6-talker babble. Test words with sparse visual neighborhoods were recognized more accurately than words with dense neighborhoods in a vision-only condition. Densities of both the acoustic and visual neighborhoods as well as their intersection overlap were predictive of how well the test words were recognized in the auditory-visual condition. These results suggest that visual neighborhoods exist and that they affect auditory-visual speech perception. One implication is that in the presence of dual sensory impairment, the boundaries of both acoustic and visual neighborhoods may shift, adversely affecting speech recognition.

Audiovisual Integration and Lipreading Abilities of Older Adults with Normal and Impaired Hearing

OBJECTIVE

The purpose of the current study was to examine how age-related hearing impairment affects lipreading and auditory-visual integration. The working hypothesis for the investigation was that presbycusic hearing loss would increase reliance on visual speech information, resulting in better lipreading and auditory-visual integration in older persons who have hearing impairment, compared with older persons who have normal hearing.

DESIGN

This study compared the performance of 53 adults with normal hearing (above age 65) and 24 adults with mild-to-moderate hearing impairment (above age 65) on auditory-only (A), visual-only (V), and auditory-visual (AV) speech perception, using consonants, words, and sentences as stimuli. All testing was conducted in the presence of multi-talker background babble, set individually for each participant and each type of stimulus, to obtain approximately equivalent A performance across the groups. In addition, we compared the two groups of participants on measures of auditory enhancement, visual enhancement, and auditory-visual integration that were derived from the A, V and AV performance scores.

RESULTS

In general, the two groups of participants performed similarly on measures of V and AV speech perception. The one exception to this finding was that the participants with hearing impairment performed significantly better than the participants with normal hearing on V identification of words. Measures of visual enhancement, auditory enhancement, and auditory-visual integration did not differ as a function of hearing status.

CONCLUSIONS

Overall, the results of the current study suggest that despite increased reliance on visual speech information, older adults who have hearing impairment do not exhibit better V speech perception or auditory-visual integration than age-matched individuals who have normal hearing. These findings indicate that inclusion of V and AV speech perception measures can provide important information for designing maximally effective audiological rehabilitation strategies.

Integration efficiency for speech perception within and across sensory modalities by normal-hearing and hearing-impaired individuals

In face-to-face speech communication, the listener extracts and integrates information from the acoustic and optic speech signals. Integration occurs within the auditory modality (i.e., across the acoustic frequency spectrum) and across sensory modalities (i.e., across the acoustic and optic signals). The difficulties experienced by some hearing-impaired listeners in understanding speech could be attributed to losses in the extraction of speech information, the integration of speech cues, or both. The present study evaluated the ability of normal-hearing and hearing-impaired listeners to integrate speech information within and across sensory modalities in order to determine the degree to which integration efficiency may be a factor in the performance of hearing-impaired listeners. Auditory-visual nonsense syllables consisting of eighteen medial consonants surrounded by the vowel [a] were processed into four nonoverlapping acoustic filter bands between 300 and 6000 Hz. A variety of one, two, three, and four filter-band combinations were presented for identification in auditory-only and auditory-visual conditions: A visual-only condition was also included. Integration efficiency was evaluated using a model of optimal integration. Results showed that normal-hearing and hearing-impaired listeners integrated information across the auditory and visual sensory modalities with a high degree of efficiency, independent of differences in auditory capabilities. However, across-frequency integration for auditory-only input was less efficient for hearing-impaired listeners. These individuals exhibited particular difficulty extracting information from the highest frequency band (4762-6000 Hz) when speech information was presented concurrently in the next lower-frequency band (1890-2381 Hz). Results suggest that integration of speech information within the auditory modality, but not across auditory and visual modalities, affects speech understanding in hearing-impaired listeners.

Audiovisual speech perception in elderly cochlear implant recipients

OBJECTIVES/HYPOTHESIS

This study examined the speech perception skills of a younger and older group of cochlear implant recipients to determine the benefit that auditory and visual information provides for speech understanding.

STUDY DESIGN

Retrospective review.

METHODS

Pre- and postimplantation speech perception scores from the Consonant-Nucleus-Consonant (CNC), the Hearing In Noise sentence Test (HINT), and the City University of New York (CUNY) tests were analyzed for 34 postlingually deafened adult cochlear implant recipients. Half were elderly (i.e., >65 y old) and other half were middle aged (i.e., 39-53 y old). The CNC and HINT tests were administered using auditory-only presentation; the CUNY test was administered using auditory-only, vision-only, and audiovisual presentation conditions

RESULTS

No differences were observed between the two age groups on the CNC and HINT tests. For a subset of individuals tested with the CUNY sentences, we found that the preimplantation speechreading scores of the younger group correlated negatively with auditory-only postimplant performance. Additionally, older individuals demonstrated a greater reliance on the integration of auditory and visual information to understand sentences than did the younger group

CONCLUSIONS

On average, the auditory-only speech perception performance of older cochlear implant recipients was similar to the performance of younger adults. However, variability in speech perception abilities was observed within and between both age groups. Differences in speechreading skills between the younger and older individuals suggest that visual speech information is processed in a different manner for elderly individuals than it is for younger adult cochlear implant recipients.