Eliciting Naturalistic Conversations: A Method for Assessing Communication Ability, Subjective Experience, and the Impacts of Noise and Hearing Impairment

Conversational Speech Behaviors Are Context Dependent

PURPOSE

According to the interpersonal synergy model of spoken dialogue, interlocutors modify their communicative behaviors to meet the contextual demands of a given conversation. Although a growing body of research supports this postulation for linguistic behaviors (e.g., semantics, syntax), little is understood about how this model applies to speech behaviors (e.g., speech rate, pitch). The purpose of this study is to test the hypothesis that interlocutors adjust their speech behaviors across different conversational tasks with different conversational goals.

METHOD

In this study, 28 participants each engaged in two different types of conversations (i.e., relational and informational) with two partners (i.e., Partner 1 and Partner 2), yielding a total of 112 conversations. We compared six acoustic measures of participant speech behavior across conversational task and partner.

RESULTS

Linear mixed-effects models demonstrated significant differences between speech feature measures in informational and relational conversations. Furthermore, these findings were generally robust across conversations with different partners.

CONCLUSIONS

Results suggest that contextual demands influence speech behaviors. These findings provide empirical support for the interpersonal synergy model and highlight important considerations for assessing speech behaviors in individuals with communication disorders.

Is speech intelligibility what speech intelligibility tests test?

Natural, conversational speech signals contain sources of symbolic and iconic information, both of which are necessary for the full understanding of speech. But speech intelligibility tests, which are generally derived from written language, present only symbolic information sources, including lexical semantics and syntactic structures. Speech intelligibility tests exclude almost all sources of information about talkers, including their communicative intentions and their cognitive states and processes. There is no reason to suspect that either hearing impairment or noise selectively affect perception of only symbolic information. We must therefore conclude that diagnosis of good or poor speech intelligibility on the basis of standard speech tests is based on measurement of only a fraction of the task of speech perception. This paper presents a descriptive comparison of information sources present in three widely used speech intelligibility tests and spontaneous, conversational speech elicited using a referential communication task. The aim of this comparison is to draw attention to the differences in not just the signals, but the tasks of listeners perceiving these different speech signals and to highlight the implications of these differences for the interpretation and generalizability of speech intelligibility test results.

Measuring Speech Intelligibility and Hearing-Aid Benefit Using Everyday Conversational Sentences in Real-World Environments

Laboratory and clinical-based assessments of speech intelligibility must evolve to better predict real-world speech intelligibility. One way of approaching this goal is to develop speech intelligibility tasks that are more representative of everyday speech communication outside the laboratory. Here, we evaluate speech intelligibility using both a standard sentence recall task based on clear, read speech (BKB sentences), and a sentence recall task consisting of spontaneously produced speech excised from conversations which took place in realistic background noises (ECO-SiN sentences). The sentences were embedded at natural speaking levels in six realistic background noises that differed in their overall level, which resulted in a range of fixed signal-to-noise ratios. Ten young, normal hearing participants took part in the study, along with 20 older participants with a range of levels of hearing loss who were tested with and without hearing-aid amplification. We found that scores were driven by hearing loss and the characteristics of the background noise, as expected, but also strongly by the speech materials. Scores obtained with the more realistic sentences were generally lower than those obtained with the standard sentences, which reduced ceiling effects for the majority of environments/listeners (but introduced floor effects in some cases). Because ceiling and floor effects limit the potential for observing changes in performance, benefits of amplification were highly dependent on the speech materials for a given background noise and participant group. Overall, the more realistic speech task offered a better dynamic range for capturing individual performance and hearing-aid benefit across the range of real-world environments we examined.

Listening-Based Communication Ability in Adults With Hearing Loss: A Scoping Review of Existing Measures

Introduction

Hearing loss in adults has a pervasive impact on health and well-being. Its effects on everyday listening and communication can directly influence participation across multiple spheres of life. These impacts, however, remain poorly assessed within clinical settings. Whilst various tests and questionnaires that measure listening and communication abilities are available, there is a lack of consensus about which measures assess the factors that are most relevant to optimising auditory rehabilitation. This study aimed to map current measures used in published studies to evaluate listening skills needed for oral communication in adults with hearing loss.

Methods

A scoping review was conducted using systematic searches in Medline, EMBASE, Web of Science and Google Scholar to retrieve peer-reviewed articles that used one or more linguistic-based measure necessary to oral communication in adults with hearing loss. The range of measures identified and their frequency where charted in relation to auditory hierarchies, linguistic domains, health status domains, and associated neuropsychological and cognitive domains.

Results

9121 articles were identified and 2579 articles that reported on 6714 discrete measures were included for further analysis. The predominant linguistic-based measure reported was word or sentence identification in quiet (65.9%). In contrast, discourse-based measures were used in 2.7% of the articles included. Of the included studies, 36.6% used a self-reported instrument purporting to measures of listening for communication. Consistent with previous studies, a large number of self-reported measures were identified (n = 139), but 60.4% of these measures were used in only one study and 80.7% were cited five times or fewer.

Discussion

Current measures used in published studies to assess listening abilities relevant to oral communication target a narrow set of domains. Concepts of communicative interaction have limited representation in current measurement. The lack of measurement consensus and heterogeneity amongst the assessments limit comparisons across studies. Furthermore, extracted measures rarely consider the broader linguistic, cognitive and interactive elements of communication. Consequently, existing measures may have limited clinical application if assessing the listening-related skills required for communication in daily life, as experienced by adults with hearing loss.

The Effects of Hearing-Aid Amplification and Noise on Conversational Dynamics Between Normal-Hearing and Hearing-Impaired Talkers

There is a long-standing tradition to assess hearing-aid benefits using lab-based speech intelligibility tests. Towards a more everyday-like scenario, the current study investigated the effects of hearing-aid amplification and noise on face-to-face communication between two conversational partners. Eleven pairs, consisting of a younger normal-hearing (NH) and an older hearing-impaired (HI) participant, solved spot-the-difference tasks while their conversations were recorded. In a two-block randomized design, the tasks were solved in quiet or noise, both with and without the HI participant receiving hearing-aid amplification with active occlusion cancellation. In the presence of 70 dB SPL babble noise, participants had fewer, slower, and less well-timed turn-starts, while speaking louder with longer inter-pausal units (IPUs, stretches of continuous speech surrounded by silence) and reducing their articulation rates. All these changes are indicative of increased communication effort. The timing of turn-starts by the HI participants exhibited more variability than that of their NH conversational partners. In the presence of background noise, the timing of turn-starts by the HI participants became even more variable, and their NH partners spoke louder. When the HI participants were provided with hearing-aid amplification, their timing of turn-starts became faster, they increased their articulation rate, and they produced shorter IPUs, all indicating reduced communication effort. In conclusion, measures of the conversational dynamics showed that background noise increased the communication effort, especially for the HI participants, and that providing hearing-aid amplification caused the HI participant to behave more like their NH conversational partner, especially in quiet situations.

Effect of Test Realism on Speech-in-noise Outcomes in Bilateral Cochlear Implant Users

OBJECTIVES

First, to evaluate the effect of laboratory-based test realism on speech intelligibility outcomes of cochlear implant users. Second, to conduct an exploratory investigation of speech intelligibility of cochlear implant users, including bilateral benefit, under realistic laboratory conditions.

DESIGN

For the first goal, the authors measured speech intelligibility scores of 15 bilateral cochlear implant recipients under three different test realism levels at two different signal-to-noise ratios (SNRs). The levels included (1) standard Bamford-Kowal-Bench-like sentences with spatially separated standard babble noise; (2) standard Bamford-Kowal-Bench-like sentences with three-dimensional recordings of actual situations; and (3) a variation of the second realism level where the sentences were obtained from natural effortful conversations. For the second goal, speech intelligibility of the realistic speech material was measured in six different acoustic scenes with realistic signal-to-noise ratios ranging from -5.8 dB to 3.2 dB.

RESULTS

Speech intelligibility was consistently highest in the most artificial (standard) test and lowest in the most realistic test. The effect of the realistic noise and that of the realistic speech material resulted in distinct SNR-dependent performance shifts with respect to their baselines. Speech intelligibility in realistic laboratory conditions was in general low, with mean scores around 60% at the highest SNR. Bilateral benefit provided on average a 7% benefit over unilateral speech understanding in the better-performing ear.

CONCLUSIONS

The results obtained here suggest that standard speech-in-noise tests overestimate the performance of cochlear implant recipients in the real world. To address this limitation, future assessments need to improve the realism over current tests by considering the realism of both, the speech and the noise materials. Likewise, speech intelligibility data under realistic conditions suggest that, insofar as these results can be considered representative of real-life performance, conversational speech and noise levels common to cochlear implant recipients are challenging in terms of speech intelligibility, with average scores around 60%. The findings and limitations are discussed alongside the factors affecting speech intelligibility.

Effect of voice support level and spectrum on conversational speech

One's own voice (autophony) is transmitted to the ears as direct airborne sound, bone conduction, and indirect airborne sound from reflections characterized by overall gain and spectro-temporal features. This study investigates how the spectral profile and gain of simulated indirect airborne sound, quantified as voice support (ST_V), affect the speaking voice of talkers. Pairs of participants performed a conversation elicitation task in anechoic conditions. The indirect airborne sound was provided in real-time via open headphones that maintain the direct airborne transmission path. Experimental conditions included high-pass, low-pass, and all-pass versions of ST_V, each presented at three overall gains, and a Baseline condition with no electroacoustic contribution to ST_V. The results show an overall speech level reduction of 0.22 dB for every additional dB of speech-weighted ST_V, i.e., a -0.22 dB/dB slope. There was some effect of ST_V spectrum on speech: slope for the high-pass condition was steeper (statistically significant) and significantly different from the all-pass slope; spectral balance (2-4 kHz vs 0-2 kHz) of speech showed an interaction effect between gender and experimental conditions. This paper's findings may inform acoustic treatments in environments where overall sound reduction is of interest for favorable ergonomics and occupational health for voice professionals.

Listening Effort Is Not the Same as Speech Intelligibility Score

Listening effort is a valuable and important notion to measure because it is among the primary complaints of people with hearing loss. It is tempting and intuitive to accept speech intelligibility scores as a proxy for listening effort, but this link is likely oversimplified and lacks actionable explanatory power. This study was conducted to explain the mechanisms of listening effort that are not captured by intelligibility scores, using sentence-repetition tasks where specific kinds of mistakes were prospectively planned or analyzed retrospectively. Effort measured as changes in pupil size among 20 listeners with normal hearing and 19 listeners with cochlear implants. Experiment 1 demonstrates that mental correction of misperceived words increases effort even when responses are correct. Experiment 2 shows that for incorrect responses, listening effort is not a function of the proportion of words correct but is rather driven by the types of errors, position of errors within a sentence, and the need to resolve ambiguity, reflecting how easily the listener can make sense of a perception. A simple taxonomy of error types is provided that is both intuitive and consistent with data from these two experiments. The diversity of errors in these experiments implies that speech perception tasks can be designed prospectively to elicit the mistakes that are more closely linked with effort. Although mental corrective action and number of mistakes can scale together in many experiments, it is possible to dissociate them to advance toward a more explanatory (rather than correlational) account of listening effort.

Effects of Noise and Second Language on Conversational Dynamics in Task Dialogue

This study provides a framework for measuring conversational dynamics between conversational partners (interlocutors). Conversations from 20 pairs of young, normal-hearing, native-Danish talkers were recorded when speaking in both quiet and noise (70 dBA sound pressure level [SPL]) and in Danish and English. Previous studies investigating the intervals from when one talker stops talking to when the next one starts, termed floor-transfer offsets (FTOs), suggest that typical turn-taking requires interlocutors to predict when the current talker will finish their turn. We hypothesized that adding noise and/or speaking in a second language (L2) would increase the communication difficulty and result in longer and more variable FTOs. The median and interquartile range of FTOs increased slightly in noise, and in L2, there was a small increase in interquartile range but a small decrease in the median of FTO durations. It took the participants longer to complete the task in both L2 and noise, indicating increased communication difficulty. The average duration of interpausal units, that is, units of connected speech surrounded by silences of 180 ms or more, increased by 18% in noise and 8% in L2. These findings suggest that talkers held their turn for longer, allowing more time for speech understanding and planning. In L2, participants spoke slower, and in both L2 and noise, they took fewer turns. These changes in behavior may have offset some of the increased difficulty when communicating in noise or L2. We speculate that talkers prioritize the maintenance of turn-taking timing over other speech measures.

The Virtual Reality Lab: Realization and Application of Virtual Sound Environments

To assess perception with and performance of modern and future hearing devices with advanced adaptive signal processing capabilities, novel evaluation methods are required that go beyond already established methods. These novel methods will simulate to a certain extent the complexity and variability of acoustic conditions and acoustic communication styles in real life. This article discusses the current state and the perspectives of virtual reality technology use in the lab for designing complex audiovisual communication environments for hearing assessment and hearing device design and evaluation. In an effort to increase the ecological validity of lab experiments, that is, to increase the degree to which lab data reflect real-life hearing-related function, and to support the development of improved hearing-related procedures and interventions, this virtual reality lab marks a transition from conventional (audio-only) lab experiments to the field. The first part of the article introduces and discusses the notion of the communication loop as a theoretical basis for understanding the factors that are relevant for acoustic communication in real life. From this, requirements are derived that allow an assessment of the extent to which a virtual reality lab reflects these factors, and which may be used as a proxy for ecological validity. The most important factor of real-life communication identified is a closed communication loop among the actively behaving participants. The second part of the article gives an overview of the current developments towards a virtual reality lab at Oldenburg University that aims at interactive and reproducible testing of subjects with and without hearing devices in challenging communication conditions. The extent to which the virtual reality lab in its current state meets the requirements defined in the first part is discussed, along with its limitations and potential further developments. Finally, data are presented from a qualitative study that compared subject behavior and performance in two audiovisual environments presented in the virtual reality lab-a street and a cafeteria-with the corresponding field environments. The results show similarities and differences in subject behavior and performance between the lab and the field, indicating that the virtual reality lab in its current state marks a step towards more ecological validity in lab-based hearing and hearing device research, but requires further development towards higher levels of ecological validity.

Hearing Aid Amplification Reduces Communication Effort of People With Hearing Impairment and Their Conversation Partners

Objectives This study investigates the hypothesis that hearing aid amplification reduces effort within conversation for both hearing aid wearers and their communication partners. Levels of effort, in the form of speech production modifications, required to maintain successful spoken communication in a range of acoustic environments are compared to earlier reported results measured in unaided conversation conditions. Design Fifteen young adult normal-hearing participants and 15 older adult hearing-impaired participants were tested in pairs. Each pair consisted of one young normal-hearing participant and one older hearing-impaired participant. Hearing-impaired participants received directional hearing aid amplification, according to their audiogram, via a master hearing aid with gain provided according to the NAL-NL2 fitting formula. Pairs of participants were required to take part in naturalistic conversations through the use of a referential communication task. Each pair took part in five conversations, each of 5-min duration. During each conversation, participants were exposed to one of five different realistic acoustic environments presented through highly open headphones. The ordering of acoustic environments across experimental blocks was pseudorandomized. Resulting recordings of conversational speech were analyzed to determine the magnitude of speech modifications, in terms of vocal level and spectrum, produced by normal-hearing talkers as a function of both acoustic environment and the degree of high-frequency average hearing impairment of their conversation partner. Results The magnitude of spectral modifications of speech produced by normal-hearing talkers during conversations with aided hearing-impaired interlocutors was smaller than the speech modifications observed during conversations between the same pairs of participants in the absence of hearing aid amplification. Conclusions The provision of hearing aid amplification reduces the effort required to maintain communication in adverse conditions. This reduction in effort provides benefit to hearing-impaired individuals and also to the conversation partners of hearing-impaired individuals. By considering the impact of amplification on both sides of dyadic conversations, this approach contributes to an increased understanding of the likely impact of hearing impairment on everyday communication.

Hearing Impairment Increases Communication Effort During Conversations in Noise

Purpose This article describes patterns of speech modifications produced by talkers as a function of the degree of hearing impairment of communication partners during naturalistic conversations in noise. An explanation of observed speech modifications is proposed in terms of a generalization of the concept of effort. This account complements existing theories of listening effort by extending the concept of effort to the domain of interactive communication. Method Twenty young adult normal hearing participants and 20 older adult hearing-impaired participants were tested in pairs. Each pair consisted of 1 young normal hearing participant and 1 older hearing-impaired participant. Pairs of participants took part in naturalistic conversations through the use of a referential communication task. Each pair completed a 5-min conversation in each of 5 different realistic acoustic environments. Results Talkers modified their speech, in terms of level and spectrum, in a gradient manner reflecting both the acoustic environment and the degree of hearing impairment of their conversation partner. All pairs of participants were able to maintain communication across all acoustic environments regardless of degree of hearing impairment and the level of environmental noise. Contrasting effects of noise and hearing impairment on speech production revealed distinct patterns of speech modifications produced by normal hearing and hearing-impaired talkers during conversation. This may reflect the fact that only the speech modifications produced by normal hearing talkers functioned to compensate for the hearing impairment of a conversation partner. Conclusions The data presented support the concept of communication effort as a dynamic feedback system between conversation participants. Additionally, these results provide insight into the nature of realistic speech signals, which are encountered by people with hearing impairment in everyday communication scenarios.