Benefit of binaural listening as revealed by speech intelligibility and listening effort

Impact of Reverberation on Speech Perception in Noise in Bimodal/Bilateral Cochlear Implant Users with and without Residual Hearing

(1) Background: The aim of the present study was to assess the impact of reverberation on speech perception in noise and spatial release from masking (SRM) in bimodal or bilateral cochlear implant (CI) users and CI subjects with low-frequency residual hearing using combined electric-acoustic stimulation (EAS). (2) Methods: In total, 10 bimodal, 14 bilateral CI users and 14 EAS users, and 17 normal hearing (NH) controls, took part in the study. Speech reception thresholds (SRTs) in unmodulated noise were assessed in co-located masker condition (S0N0) with a spatial separation of speech and noise (S0N60) in both free-field and loudspeaker-based room simulation for two different reverberation times. (3) Results: There was a significant detrimental effect of reverberation on SRTs and SRM in all subject groups. A significant difference between the NH group and all the CI/EAS groups was found. There was no significant difference in SRTs between any CI and EAS group. Only NH subjects achieved spatial release from masking in reverberation, whereas no beneficial effect of spatial separation of speech and noise was found in any CI/EAS group. (4) Conclusions: The subject group with electric-acoustic stimulation did not yield a superior outcome in terms of speech perception in noise under reverberation when the noise was presented towards the better hearing ear.

Electrophysiological Measures of Listening-in-Noise With and Without Remote Microphone System Use in Autistic and Non-Autistic Youth

OBJECTIVES

This study examined the neural mechanisms by which remote microphone (RM) systems might lead to improved behavioral performance on listening-in-noise tasks in autistic and non-autistic youth.

DESIGN

Cortical auditory evoked potentials (CAEPs) were recorded in autistic (n = 25) and non-autistic (n = 22) youth who were matched at the group level on chronological age ( M = 14.21 ± 3.39 years) and biological sex. Potentials were recorded during an active syllable identification task completed in quiet and in multi-talker babble noise with and without the use of an RM system. The effects of noise and RM system use on speech-sound-evoked P1-N1-P2 responses and the associations between the cortical responses and behavioral performance on syllable identification were examined.

RESULTS

No group differences were observed for behavioral or CAEP measures of speech processing in quiet or in noise. In the combined sample, syllable identification in noise was less accurate and slower than in the quiet condition. The addition of the RM system to the noise condition restored accuracy, but not the response speed, to the levels observed in quiet. The CAEP analyses noted amplitude reductions and latency delays in the noise compared with the quiet condition. The RM system use increased the N1 amplitude as well as reduced and delayed the P2 response relative to the quiet and noise conditions. Exploratory brain-behavior correlations revealed that larger N1 amplitudes in the RM condition were associated with greater behavioral accuracy of syllable identification. Reduced N1 amplitude and accelerated P2 response were associated with shorter syllable identification response times when listening with the RM system.

CONCLUSIONS

Findings suggest that although listening-in-noise with an RM system might remain effortful, the improved signal to noise ratio facilitates attention to the sensory features of the stimuli and increases speech sound identification accuracy.

Children's use of spatial and visual cues for release from perceptual masking

This study examined the role of visual speech in providing release from perceptual masking in children by comparing visual speech benefit across conditions with and without a spatial separation cue. Auditory-only and audiovisual speech recognition thresholds in a two-talker speech masker were obtained from 21 children with typical hearing (7-9 years of age) using a color-number identification task. The target was presented from a loudspeaker at 0° azimuth. Masker source location varied across conditions. In the spatially collocated condition, the masker was also presented from the loudspeaker at 0° azimuth. In the spatially separated condition, the masker was presented from the loudspeaker at 0° azimuth and a loudspeaker at -90° azimuth, with the signal from the -90° loudspeaker leading the signal from the 0° loudspeaker by 4 ms. The visual stimulus (static image or video of the target talker) was presented at 0° azimuth. Children achieved better thresholds when the spatial cue was provided and when the visual cue was provided. Visual and spatial cue benefit did not differ significantly depending on the presence of the other cue. Additional studies are needed to characterize how children's preferential use of visual and spatial cues varies depending on the strength of each cue.

Effects of Critical Distance and Reverberation on Listening Effort in Adults

PURPOSE

Mixed historical data on how listening effort is affected by reverberation and listener-to-speaker distance challenge existing models of listening effort. This study investigated the effects of reverberation and listener-to-speaker distance on behavioral and subjective measures of listening effort: (a) when listening at a fixed signal-to-noise ratio (SNR) and (b) at SNRs that were manipulated so that word recognition would be comparable across different reverberation times and listening distances. It was expected that increased reverberation would increase listening effort but only when listening outside critical distance.

METHOD

Nineteen adults (21-40 years) with no hearing loss completed a dual-task paradigm. The primary task was word recognition and the secondary task was timed word categorization; response times indexed behavioral listening effort. Additionally, participants provided subjective ratings in each condition. Testing was completed at two reverberation levels (moderate and high, RT₃₀ = 469 and 1,223 ms, respectively) and at two listener-to-speaker distances (inside and outside critical distance for the test room, 1.25 and 4 m, respectively).

RESULTS

Increased reverberation and listening distances worsened word recognition performance and both behavioral and subjective listening effort. The effect of reverberation was exacerbated when listeners were outside critical distance. Subjective experience of listening effort persisted even when word recognition was comparable across conditions.

CONCLUSIONS

Longer reverberation times or listening outside the room's critical distance negatively affected behavioral and subjective listening effort. This study extends understanding of listening effort in reverberant rooms by highlighting the effect of listener's position relative to the room's critical distance.

Assessing and Modeling Spatial Release From Listening Effort in Listeners With Normal Hearing: Reference Ranges and Effects of Noise Direction and Age

Listening to speech in noisy environments is challenging and effortful. Factors like the signal-to-noise ratio (SNR), the spatial separation between target speech and noise interferer(s), and possibly also the listener's age might influence perceived listening effort (LE). This study measured and modeled the effect of the spatial separation of target speech and interfering stationary speech-shaped noise on the perceived LE and its relation to the age of the listeners. Reference ranges for the relationship between subjectively perceived LE and SNR for different noise azimuths were established. For this purpose, 70 listeners with normal hearing and from three age groups rated the perceived LE using the Adaptive Categorical Listening Effort Scaling method (ACALES, Krueger et al., 2017a) with speech from the front and noise from 0°, 90°, 135°, or 180° azimuth. Based on these data, the spatial release from listening effort (SRLE) was calculated. The noise azimuth had a strong effect on SRLE, with the highest release for 135°. The binaural speech intelligibility model (BSIM2020, Hauth et al., 2020) predicted SRLE very well at negative SNRs, but overestimated for positive SNRs. No significant effect of age was found on the respective subjective ratings. Therefore, the reference ranges were determined independently of age. These reference ranges can be used for the classification of LE measurements. However, when the increase of the perceived LE with SNR was analyzed, a significant age difference was found between the listeners of the youngest and oldest group when considering the upper range of the LE function.

A biologically oriented algorithm for spatial sound segregation

Listening in an acoustically cluttered scene remains a difficult task for both machines and hearing-impaired listeners. Normal-hearing listeners accomplish this task with relative ease by segregating the scene into its constituent sound sources, then selecting and attending to a target source. An assistive listening device that mimics the biological mechanisms underlying this behavior may provide an effective solution for those with difficulty listening in acoustically cluttered environments (e.g., a cocktail party). Here, we present a binaural sound segregation algorithm based on a hierarchical network model of the auditory system. In the algorithm, binaural sound inputs first drive populations of neurons tuned to specific spatial locations and frequencies. The spiking response of neurons in the output layer are then reconstructed into audible waveforms via a novel reconstruction method. We evaluate the performance of the algorithm with a speech-on-speech intelligibility task in normal-hearing listeners. This two-microphone-input algorithm is shown to provide listeners with perceptual benefit similar to that of a 16-microphone acoustic beamformer. These results demonstrate the promise of this biologically inspired algorithm for enhancing selective listening in challenging multi-talker scenes.

A joint framework for blind prediction of binaural speech intelligibility and perceived listening effort

Speech perception is strongly affected by noise and reverberation in the listening room, and binaural processing can substantially facilitate speech perception in conditions when target speech and maskers originate from different directions. Most studies and proposed models for predicting spatial unmasking have focused on speech intelligibility. The present study introduces a model framework that predicts both speech intelligibility and perceived listening effort from the same output measure. The framework is based on a combination of a blind binaural processing stage employing a blind equalization cancelation (EC) mechanism, and a blind backend based on phoneme probability classification. Neither frontend nor backend require any additional information, such as the source directions, the signal-to-noise ratio (SNR), or the number of sources, allowing for a fully blind perceptual assessment of binaural input signals consisting of target speech mixed with noise. The model is validated against a recent data set in which speech intelligibility and perceived listening effort were measured for a range of acoustic conditions differing in reverberation and binaural cues [Rennies and Kidd (2018), J. Acoust. Soc. Am. 144, 2147-2159]. Predictions of the proposed model are compared with a non-blind binaural model consisting of a non-blind EC stage and a backend based on the speech intelligibility index. The analyses indicated that all main trends observed in the experiments were correctly predicted by the blind model. The overall proportion of variance explained by the model (R² = 0.94) for speech intelligibility was slightly worse than for the non-blind model (R² = 0.98). For listening effort predictions, both models showed lower prediction accuracy, but still explained significant proportions of the observed variance (R² = 0.88 and R² = 0.71 for the non-blind and blind model, respectively). Closer inspection showed that the differences between data and predictions were largest for binaural conditions at high SNRs, where the perceived listening effort of human listeners tended to be underestimated by the models, specifically by the blind version.

Effects of degraded speech processing and binaural unmasking investigated using functional near-infrared spectroscopy (fNIRS)

The present study aimed to investigate the effects of degraded speech perception and binaural unmasking using functional near-infrared spectroscopy (fNIRS). Normal hearing listeners were tested when attending to unprocessed or vocoded speech, presented to the left ear at two speech-to-noise ratios (SNRs). Additionally, by comparing monaural versus diotic masker noise, we measured binaural unmasking. Our primary research question was whether the prefrontal cortex and temporal cortex responded differently to varying listening configurations. Our a priori regions of interest (ROIs) were located at the left dorsolateral prefrontal cortex (DLPFC) and auditory cortex (AC). The left DLPFC has been reported to be involved in attentional processes when listening to degraded speech and in spatial hearing processing, while the AC has been reported to be sensitive to speech intelligibility. Comparisons of cortical activity between these two ROIs revealed significantly different fNIRS response patterns. Further, we showed a significant and positive correlation between self-reported task difficulty levels and fNIRS responses in the DLPFC, with a negative but non-significant correlation for the left AC, suggesting that the two ROIs played different roles in effortful speech perception. Our secondary question was whether activity within three sub-regions of the lateral PFC (LPFC) including the DLPFC was differentially affected by varying speech-noise configurations. We found significant effects of spectral degradation and SNR, and significant differences in fNIRS response amplitudes between the three regions, but no significant interaction between ROI and speech type, or between ROI and SNR. When attending to speech with monaural and diotic noises, participants reported the latter conditions being easier; however, no significant main effect of masker condition on cortical activity was observed. For cortical responses in the LPFC, a significant interaction between SNR and masker condition was observed. These findings suggest that binaural unmasking affects cortical activity through improving speech reception threshold in noise, rather than by reducing effort exerted.

Spatial release from masking in reverberation for school-age children

Understanding speech in noisy environments, such as classrooms, is a challenge for children. When a spatial separation is introduced between the target and masker, as compared to when both are co-located, children demonstrate intelligibility improvement of the target speech. Such intelligibility improvement is known as spatial release from masking (SRM). In most reverberant environments, binaural cues associated with the spatial separation are distorted; the extent to which such distortion will affect children's SRM is unknown. Two virtual acoustic environments with reverberation times between 0.4 s and 1.1 s were compared. SRM was measured using a spatial separation with symmetrically displaced maskers to maximize access to binaural cues. The role of informational masking in modulating SRM was investigated through voice similarity between the target and masker. Results showed that, contradictory to previous developmental findings on free-field SRM, children's SRM in reverberation has not yet reached maturity in the 7-12 years age range. When reducing reverberation, an SRM improvement was seen in adults but not in children. Our findings suggest that, even though school-age children have access to binaural cues that are distorted in reverberation, they demonstrate immature use of such cues for speech-in-noise perception, even in mild reverberation.

Relation between hearing abilities and preferred playback settings for speech perception in complex listening conditions

OBJECTIVE

This study investigated if individual preferences with respect to the trade-off between a good signal-to-noise ratio and a distortion-free speech target were stable across different masking conditions and if simple adjustment methods could be used to identify subjects as either "noise haters" or "distortions haters".

DESIGN

In each masking condition, subjects could adjust the target speech level according to their preferences by employing (i) linear gain or gain at the cost of (ii) clipping distortions or (iii) compression distortions. The comparison of these processing conditions allowed investigating the preferred trade-off between distortions and noise disturbance.

STUDY SAMPLE

Thirty subjects differing widely in hearing status (normal-hearing to moderately impaired) and age (23-85 years).

RESULTS

High test-retest stability of individual preferences was found for all modification schemes. The preference adjustments suggested that subjects could be consistently categorised along a scale from "noise haters" to "distortion haters", and this preference trait remained stable through all maskers, spatial conditions, and types of distortions.

CONCLUSIONS

Employing quick self-adjustment to collect listening preferences in complex listening conditions revealed a stable preference trait along the "noise vs. distortions" tolerance dimension. This could potentially help in fitting modern hearing aid algorithms to the individual user.

Development and Critical Evaluation of a Condition-Specific Preference-Based Measure Sensitive to Binaural Hearing in Adults: The York Binaural Hearing-Related Quality-of-Life System

OBJECTIVES

The widely-used generic preference-based measures of health-related quality of life-the EuroQol Descriptive System (EQ-5D) and the Health Utilities Index (HUI3)-are limited in their response to technologies that improve hearing. The EQ-5D lacks construct validity for hearing, while the HUI3 is restricted by a ceiling effect and by using speech reception as the only evidence of the ability to hear. Consequently, neither measure consistently registers benefits from binaural hearing, such as those from bilateral versus unilateral cochlear implantation. The objectives were to test whether informants value binaural hearing, to develop a condition-specific preference-based measure sensitive to binaural hearing, to assess the psychometric properties of the new instrument, and to determine whether it meets requirements for informing judgments of cost-effectiveness: does it measure greater gains than do the generic preference-based measures, while avoiding exaggerating losses, and displaying sensitivity to side effects?

DESIGN

Three levels of function, ranging from no difficulty to great difficulty, were defined on each of three dimensions where listening is easier or more successful when hearing is binaural rather than monaural: perception of speech in spatially separated noise, localization of sounds, and effort and fatigue. Informants (N = 203) valued the 27 combinations of levels and dimensions in a time trade-off task with a 10-year time frame to provide a value of binaural-related quality of life ("binaural utility") for each combination. A questionnaire was compiled to allow respondents to report their level of function on each dimension so that a value of binaural utility could be assigned to them. The questionnaire and the age-standardized valuations constitute The York Binaural Hearing-Related Quality-of-Life System (YBHRQL). Adult users of unilateral implants (N = 8), bilateral implants (N = 11), or bimodal aiding (N = 9) undertook performance tests of spatial listening and completed the HUI3, EQ-5D, and Speech, Spatial, and Qualities of Hearing (SSQ) questionnaires. They completed the YBHRQL questionnaire 24 and 38 mo later.

RESULTS

Despite long intervals between measurements, the YBHRQL demonstrated desirable psychometric properties: good construct validity evidenced by significant correlations with performance measures and the SSQ index; a greater ability than the EQ-5D or HUI3 to distinguish unilateral, bimodal, and bilateral listening; and good reproducibility. The YBHRQL did not exaggerate losses of utility but was insensitive to a potential side effect of implantation (pain/discomfort). It measured a gain in utility from bilateral compared with unilateral implantation (median = 0.11, interquartile range, 0.03 to 0.16) that was greater than the gain measured by the EQ-5D (0.00, 0.00 to 0.00) but not the HUI3 (0.00, 0.00 to 0.17).

CONCLUSIONS

The YBHRQL summarizes the contribution of binaural hearing to quality of life by combining the functional status of a listener with the preferences of independent informants. It would be an efficient clinical outcome measure. In addition, if used alongside the EQ-5D or HUI3, it would provide evidence which could beneficially modulate confidence in the cost-effectiveness of interventions. Further research on its sensitivity to side effects, and on the size of the gains in utility which it measures, is needed to determine whether it could stand alone to inform resource-allocation decisions.

Dichotic listening performance and effort as a function of spectral resolution and interaural symmetry

One potential benefit of bilateral cochlear implants is reduced listening effort in speech-on-speech masking situations. However, the symmetry of the input across ears, possibly related to spectral resolution, could impact binaural benefits. Fifteen young adults with normal hearing performed digit recall with target and interfering digits presented to separate ears and attention directed to the target ear. Recall accuracy and pupil size over time (used as an index of listening effort) were measured for unprocessed, 16-channel vocoded, and 4-channel vocoded digits. Recall accuracy was significantly lower for dichotic (with interfering digits) than for monotic listening. Dichotic recall accuracy was highest when the target was less degraded and the interferer was more degraded. With matched target and interferer spectral resolution, pupil dilation was lower with more degradation. Pupil dilation grew more shallowly over time when the interferer had more degradation. Overall, interferer spectral resolution more strongly affected listening effort than target spectral resolution. These results suggest that interfering speech both lowers performance and increases listening effort, and that the relative spectral resolution of target and interferer affect the listening experience. Ignoring a clearer interferer is more effortful.

Postural Control While Listening in Younger and Middle-Aged Adults

OBJECTIVES

The motivation for this research is to determine whether a listening-while-balancing task would be sensitive to quantifying listening effort in middle age. The premise behind this exploratory work is that a decrease in postural control would be demonstrated in challenging acoustic conditions, more so in middle-aged than in younger adults.

DESIGN

A dual-task paradigm was employed with speech understanding as one task and postural control as the other. For the speech perception task, participants listened to and repeated back sentences in the presence of other sentences or steady-state noise. Targets and maskers were presented in both spatially-coincident and spatially-separated conditions. The postural control task required participants to stand on a force platform either in normal stance (with feet approximately shoulder-width apart) or in tandem stance (with one foot behind the other). Participants also rated their subjective listening effort at the end of each block of trials.

RESULTS

Postural control was poorer for both groups of participants when the listening task was completed at a more adverse (vs. less adverse) signal-to-noise ratio. When participants were standing normally, postural control in dual-task conditions was negatively associated with degree of high-frequency hearing loss, with individuals who had higher pure-tone thresholds exhibiting poorer balance. Correlation analyses also indicated that reduced speech recognition ability was associated with poorer postural control in both single- and dual-task conditions. Middle-aged participants exhibited larger dual-task costs when the masker was speech, as compared to when it was noise. Individuals who reported expending greater effort on the listening task exhibited larger dual-task costs when in normal stance.

CONCLUSIONS

Listening under challenging acoustic conditions can have a negative impact on postural control, more so in middle-aged than in younger adults. One explanation for this finding is that the increased effort required to successfully listen in adverse environments leaves fewer resources for maintaining balance, particularly as people age. These results provide preliminary support for using this type of ecologically-valid dual-task paradigm to quantify the costs associated with understanding speech in adverse acoustic environments.

Binaural squelch effect in unilateral otosclerosis surgery: comparison of speech intelligibility in noise before-after surgery

PURPOSE

We aimed to evaluate the speech intelligibility benefit in noise provided by stapedotomy in the treatment of unilateral otosclerosis.

METHODS

We enrolled adults suffering from unilateral conductive hearing loss and followed them up until 9 months after surgery. The patients underwent a free field speech hearing evaluation using the French Matrix test before and after stapedotomy. Speech material was sent to the front of the patients (S0) and noise was presented either at the front (N0), or at the operated ear (N-90) or at the non-operated ear (N + 90). The speech intelligibility benefit in noise was assessed by comparing Squelch effect (SE), Head shadow effect (HS) and Binaural redundancy (BR) before and after surgery. SE was measured as the difference in speech reception thresholds (SRT) between S0N + 90 situations before and after surgery, HS as the difference in SRT between S0N + 90 and S0N-90 situations, and BR as the difference in SRT between S0N0 situations before and after surgery. In addition, two quality of life's questionnaires were completed by patients to evaluate their discomfort.

RESULTS

Among 25 patients, 19 were followed up during 9 months, 4 were excluded and 2 were lost for the following-up. Stapedotomy provided a restoration of SE of 3.7 dB SNR (p < 0.001) and a BR gain of 1.8 dB SNR (p < 0.001). HS did not show any statistical variation after surgery (p = 0.077). Finally, the questionnaires showed a residual hearing discomfort.

CONCLUSION

Stapedotomy provided a binaural benefit with the restoration of the SE and BR but which remained lower than in the normal-hearing population.

TRIAL REGISTRATION

The 07/02/2018 on Clinical.Trial.Gouv: NCT03587792.

Modeling Binaural Unmasking of Speech Using a Blind Binaural Processing Stage

The equalization cancellation model is often used to predict the binaural masking level difference. Previously its application to speech in noise has required separate knowledge about the speech and noise signals to maximize the signal-to-noise ratio (SNR). Here, a novel, blind equalization cancellation model is introduced that can use the mixed signals. This approach does not require any assumptions about particular sound source directions. It uses different strategies for positive and negative SNRs, with the switching between the two steered by a blind decision stage utilizing modulation cues. The output of the model is a single-channel signal with enhanced SNR, which we analyzed using the speech intelligibility index to compare speech intelligibility predictions. In a first experiment, the model was tested on experimental data obtained in a scenario with spatially separated target and masker signals. Predicted speech recognition thresholds were in good agreement with measured speech recognition thresholds with a root mean square error less than 1 dB. A second experiment investigated signals at positive SNRs, which was achieved using time compressed and low-pass filtered speech. The results demonstrated that binaural unmasking of speech occurs at positive SNRs and that the modulation-based switching strategy can predict the experimental results.

Measurement and Prediction of Binaural-Temporal Integration of Speech Reflections

For speech intelligibility in rooms, the temporal integration of speech reflections is typically modeled by separating the room impulse response (RIR) into an early (assumed beneficial for speech intelligibility) and a late part (assumed detrimental). This concept was challenged in this study by employing binaural RIRs with systematically varied interaural phase differences (IPDs) and amplitude of the direct sound and a variable number of reflections delayed by up to 200 ms. Speech recognition thresholds in stationary noise were measured in normal-hearing listeners for 86 conditions. The data showed that direct sound and one or several early speech reflections could be perfectly integrated when they had the same IPD. Early reflections with the same IPD as the noise (but not as the direct sound) could not be perfectly integrated with the direct sound. All conditions in which the dominant speech information was within the early RIR components could be well predicted by a binaural speech intelligibility model using classic early/late separation. In contrast, when amplitude or IPD favored late RIR components, listeners appeared to be capable of focusing on these components rather than on the precedent direct sound. This could not be modeled by an early/late separation window but required a temporal integration window that can be flexibly shifted along the RIR.

Energetic and Informational Components of Speech-on-Speech Masking in Binaural Speech Intelligibility and Perceived Listening Effort

Speech perception in complex sound fields can greatly benefit from different unmasking cues to segregate the target from interfering voices. This study investigated the role of three unmasking cues (spatial separation, gender differences, and masker time reversal) on speech intelligibility and perceived listening effort in normal-hearing listeners. Speech intelligibility and categorically scaled listening effort were measured for a female target talker masked by two competing talkers with no unmasking cues or one to three unmasking cues. In addition to natural stimuli, all measurements were also conducted with glimpsed speech-which was created by removing the time-frequency tiles of the speech mixture in which the maskers dominated the mixture-to estimate the relative amounts of informational and energetic masking as well as the effort associated with source segregation. The results showed that all unmasking cues as well as glimpsing improved intelligibility and reduced listening effort and that providing more than one cue was beneficial in overcoming informational masking. The reduction in listening effort due to glimpsing corresponded to increases in signal-to-noise ratio of 8 to 18 dB, indicating that a significant amount of listening effort was devoted to segregating the target from the maskers. Furthermore, the benefit in listening effort for all unmasking cues extended well into the range of positive signal-to-noise ratios at which speech intelligibility was at ceiling, suggesting that listening effort is a useful tool for evaluating speech-on-speech masking conditions at typical conversational levels.