Extended High Frequencies Provide Both Spectral and Temporal Information to Improve Speech-in-Speech Recognition

Factors influencing the minimum audible change in talker head orientation cues using diotic stimuli

The perception of a talker's head orientation is an ecologically relevant task. Humans are able to discriminate changes in talker head orientation using acoustic cues. Factors that may influence measures of this ability have not been well characterized. Here, we examined the minimum audible change in head orientation cues (MACHO) using diotic stimuli. The effects of several factors were tested: talker and gender, stimulus bandwidth (full-band vs low-pass filtered at 8 or 10 kHz), transducer (loudspeaker vs headphone), stimulus uncertainty (interleaved vs blocked presentation of four talkers), and vocal production mode (speech vs singing). The best performance of ∼41° was achieved for full-band, blocked presentation of speech over a loudspeaker. Greater stimulus uncertainty (interleaved presentation) worsened the MACHO by 26%. Bandlimiting at 8 and 10 kHz worsened performance by an additional 22% and 14%, respectively. At equivalent overall sound levels, performance was better for speech than for singing. There was some limited evidence for the transducer influencing the MACHO. These findings suggest the MACHO relies on multiple factors manipulated here. One of the largest, consistent effects was that of talker, suggesting head orientation cues are highly dependent on individual talker characteristics. This may be due to individual variability in speech directivity patterns.

Explainable machine learning reveals the relationship between hearing thresholds and speech-in-noise recognition in listeners with normal audiograms

Some individuals complain of listening-in-noise difficulty despite having a normal audiogram. In this study, machine learning is applied to examine the extent to which hearing thresholds can predict speech-in-noise recognition among normal-hearing individuals. The specific goals were to (1) compare the performance of one standard (GAM, generalized additive model) and four machine learning models (ANN, artificial neural network; DNN, deep neural network; RF, random forest; XGBoost; eXtreme gradient boosting), and (2) examine the relative contribution of individual audiometric frequencies and demographic variables in predicting speech-in-noise recognition. Archival data included thresholds (0.25-16 kHz) and speech recognition thresholds (SRTs) from listeners with clinically normal audiograms (n = 764 participants or 1528 ears; age, 4-38 years old). Among the machine learning models, XGBoost performed significantly better than other methods (mean absolute error; MAE = 1.62 dB). ANN and RF yielded similar performances (MAE = 1.68 and 1.67 dB, respectively), whereas, surprisingly, DNN showed relatively poorer performance (MAE = 1.94 dB). The MAE for GAM was 1.61 dB. SHapley Additive exPlanations revealed that age, thresholds at 16 kHz, 12.5 kHz, etc., on the order of importance, contributed to SRT. These results suggest the importance of hearing in the extended high frequencies for predicting speech-in-noise recognition in listeners with normal audiograms.

Predicting speech-in-speech recognition: Short-term audibility and spatial separation

Quantifying the factors that predict variability in speech-in-speech recognition represents a fundamental challenge in auditory science. Stimulus factors associated with energetic and informational masking (IM) modulate variability in speech-in-speech recognition, but energetic effects can be difficult to estimate in spectro-temporally dynamic speech maskers. The current experiment characterized the effects of short-term audibility and differences in target and masker location (or perceived location) on the horizontal plane for sentence recognition in two-talker speech. Thirty young adults with normal hearing (NH) participated. Speech reception thresholds and keyword recognition at a fixed signal-to-noise ratio (SNR) were measured in each spatial condition. Short-term audibility for each keyword was quantified using a glimpsing model. Results revealed that speech-in-speech recognition depended on the proportion of audible glimpses available in the target + masker keyword stimulus in each spatial condition, even across stimuli presented at a fixed global SNR. Short-term audibility requirements were greater for colocated than spatially separated speech-in-speech recognition, and keyword recognition improved more rapidly as a function of increases in target audibility with spatial separation. Results indicate that spatial cues enhance glimpsing efficiency in competing speech for young adults with NH and provide a quantitative framework for estimating IM for speech-in-speech recognition in different spatial configurations.

Machine Learning Methods for Small Data Challenges in Molecular Science

Small data are often used in scientific and engineering research due to the presence of various constraints, such as time, cost, ethics, privacy, security, and technical limitations in data acquisition. However, big data have been the focus for the past decade, small data and their challenges have received little attention, even though they are technically more severe in machine learning (ML) and deep learning (DL) studies. Overall, the small data challenge is often compounded by issues, such as data diversity, imputation, noise, imbalance, and high-dimensionality. Fortunately, the current big data era is characterized by technological breakthroughs in ML, DL, and artificial intelligence (AI), which enable data-driven scientific discovery, and many advanced ML and DL technologies developed for big data have inadvertently provided solutions for small data problems. As a result, significant progress has been made in ML and DL for small data challenges in the past decade. In this review, we summarize and analyze several emerging potential solutions to small data challenges in molecular science, including chemical and biological sciences. We review both basic machine learning algorithms, such as linear regression, logistic regression (LR), k-nearest neighbor (KNN), support vector machine (SVM), kernel learning (KL), random forest (RF), and gradient boosting trees (GBT), and more advanced techniques, including artificial neural network (ANN), convolutional neural network (CNN), U-Net, graph neural network (GNN), Generative Adversarial Network (GAN), long short-term memory (LSTM), autoencoder, transformer, transfer learning, active learning, graph-based semi-supervised learning, combining deep learning with traditional machine learning, and physical model-based data augmentation. We also briefly discuss the latest advances in these methods. Finally, we conclude the survey with a discussion of promising trends in small data challenges in molecular science.

Speech understanding and extended high-frequency hearing sensitivity in blast-exposed veteransa)

Auditory difficulties reported by normal-hearing Veterans with a history of blast exposure are primarily thought to stem from processing deficits in the central nervous system. However, previous work on speech understanding in noise difficulties in this patient population have only considered peripheral hearing thresholds in the standard audiometric range. Recent research suggests that variability in extended high-frequency (EHF; >8 kHz) hearing sensitivity may contribute to speech understanding deficits in normal-hearing individuals. Therefore, this work was designed to identify the effects of blast exposure on several common clinical speech understanding measures and EHF hearing sensitivity. This work also aimed to determine whether variability in EHF hearing sensitivity contributes to speech understanding difficulties in normal-hearing blast-exposed Veterans. Data from 41 normal- or near-normal-hearing Veterans with a history of blast exposure and 31 normal- or near-normal-hearing control participants with no history of head injury were employed in this study. Analysis identified an effect of blast exposure on several speech understanding measures but showed no statistically significant differences in EHF thresholds between participant groups. Data showed that variability in EHF hearing sensitivity did not contribute to group-related differences in speech understanding, although study limitations impact interpretation of these results.

Differential benefits of unmasking extended high-frequency content of target or background speech

Current evidence supports the contribution of extended high frequencies (EHFs; >8 kHz) to speech recognition, especially for speech-in-speech scenarios. However, it is unclear whether the benefit of EHFs is due to phonetic information in the EHF band, EHF cues to access phonetic information at lower frequencies, talker segregation cues, or some other mechanism. This study investigated the mechanisms of benefit derived from a mismatch in EHF content between target and masker talkers for speech-in-speech recognition. EHF mismatches were generated using full band (FB) speech and speech low-pass filtered at 8 kHz. Four filtering combinations with independently filtered target and masker speech were used to create two EHF-matched and two EHF-mismatched conditions for one- and two-talker maskers. Performance was best with the FB target and the low-pass masker in both one- and two-talker masker conditions, but the effect was larger for the two-talker masker. No benefit of an EHF mismatch was observed for the low-pass filtered target. A word-by-word analysis indicated higher recognition odds with increasing EHF energy level in the target word. These findings suggest that the audibility of target EHFs provides target phonetic information or target segregation and selective attention cues, but that the audibility of masker EHFs does not confer any segregation benefit.

Suprathreshold auditory processes in listeners with normal audiograms but extended high-frequency hearing lossa)

Hearing loss in the extended high-frequency (EHF) range (>8 kHz) is widespread among young normal-hearing adults and could have perceptual consequences such as difficulty understanding speech in noise. However, it is unclear how EHF hearing loss might affect basic psychoacoustic processes. The hypothesis that EHF hearing loss is associated with poorer auditory resolution in the standard frequencies was tested. Temporal resolution was characterized by amplitude modulation detection thresholds (AMDTs), and spectral resolution was characterized by frequency change detection thresholds (FCDTs). AMDTs and FCDTs were measured in adults with or without EHF loss but with normal clinical audiograms. AMDTs were measured with 0.5- and 4-kHz carrier frequencies; similarly, FCDTs were measured for 0.5- and 4-kHz base frequencies. AMDTs were significantly higher with the 4 kHz than the 0.5 kHz carrier, but there was no significant effect of EHF loss. There was no significant effect of EHF loss on FCDTs at 0.5 kHz; however, FCDTs were significantly higher at 4 kHz for listeners with than without EHF loss. This suggests that some aspects of auditory resolution in the standard audiometric frequency range may be compromised in listeners with EHF hearing loss despite having a normal audiogram.

Functional consequences of extended high frequency hearing impairment: Evidence from the speech, spatial, and qualities of hearing scale

Hearing loss in the extended high frequencies, despite a normal audiogram, could affect speech-in-noise recognition. However, it is not known if extended high frequency (EHF) hearing loss is associated with everyday listening and communication deficits. The present study aimed to determine the functional effects of impaired EHF hearing among adults using the Speech, Spatial, and Qualities of Hearing Scale (SSQ). A secondary objective was to evaluate the relationship between objective (speech-in-noise recognition) and subjective (SSQ) measures of hearing function. Listeners with EHF hearing loss provided lower SSQ ratings compared with their EHF-normal counterparts. The lower ratings could not be attributed to the age or standard audiogram of the listeners. Ratings from more than 50% of EHF-impaired listeners were below the 2 SD cutoff point obtained from EHF-normal listeners. The mean speech recognition threshold was poorer for EHF-impaired listeners, and a poorer speech recognition threshold was associated with lower SSQ ratings, i.e., poorer self-reported hearing ability. For some listeners, EHF hearing loss could be the missing link between self-reported hearing difficulty in the presence of a normal audiogram. These findings provide evidence for the functional deficits associated with EHF hearing loss and invoke the need for future investigations into the management of EHF loss.

On the use of the TIMIT, QuickSIN, NU-6, and other widely used bandlimited speech materials for speech perception experiments

The use of spectrally degraded speech signals deprives listeners of acoustic information that is useful for speech perception. Several popular speech corpora, recorded decades ago, have spectral degradations, including limited extended high-frequency (EHF) (>8 kHz) content. Although frequency content above 8 kHz is often assumed to play little or no role in speech perception, recent research suggests that EHF content in speech can have a significant beneficial impact on speech perception under a wide range of natural listening conditions. This paper provides an analysis of the spectral content of popular speech corpora used for speech perception research to highlight the potential shortcomings of using bandlimited speech materials. Two corpora analyzed here, the TIMIT and NU-6, have substantial low-frequency spectral degradation (<500 Hz) in addition to EHF degradation. We provide an overview of the phenomena potentially missed by using bandlimited speech signals, and the factors to consider when selecting stimuli that are sensitive to these effects.

Reconsidering commonly used stimuli in speech perception experiments

This paper examines some commonly used stimuli in speech perception experiments and raises questions about their use, or about the interpretations of previous results. The takeaway messages are: 1) the Hillenbrand vowels represent a particular dialect rather than a gold standard, and English vowels contain spectral dynamics that have been largely underappreciated, 2) the /ɑ/ context is very common but not clearly superior as a context for testing consonant perception, 3) /ɑ/ is particularly problematic when testing voice-onset-time perception because it introduces strong confounds in the formant transitions, 4) /dɑ/ is grossly overrepresented in neurophysiological studies and yet is insufficient as a generalized proxy for "speech perception," and 5) digit tests and matrix sentences including the coordinate response measure are systematically insensitive to important patterns in speech perception. Each of these stimulus sets and concepts is described with careful attention to their unique value and also cases where they might be misunderstood or over-interpreted.

Wearable Neck Surface Accelerometers for Occupational Vocal Health Monitoring: Instrument and Analysis Validation Study

BACKGROUND

Neck surface accelerometer (NSA) wearable devices have been developed for voice and upper airway health monitoring. As opposed to acoustic sounds, NSA senses mechanical vibrations propagated from the vocal tract to neck skin, which are indicative of a person's voice and airway conditions. NSA signals do not carry identifiable speech information and a speaker's privacy is thus protected, which is important and necessary for continuous wearable monitoring. Our device was already tested for its durable endurance and signal processing algorithms in controlled laboratory conditions.

OBJECTIVE

This study aims to further evaluate both instrument and analysis validity in a group of occupational vocal users, namely, voice actors, who use their voices extensively at work in an ecologically valid setting.

METHODS

A total of 16 professional voice actors (age range 21-50 years; 11 females and 5 males) participated in this study. All participants were mounted with an NSA on their sternal notches during the voice acting and voice assessment sessions. The voice acting session was 4-hour long, directed by a voice director in a professional sound studio. Voice assessment sessions were conducted before, during, and 48 hours after the acting session. The assessment included phonation tasks of passage reading, sustained vowels, maximum vowel phonation, and pitch glides. Clinical acoustic metrics (eg, fundamental frequency, cepstral measures) and a vocal dose measure (ie, accumulated distance dose from acting) were computed from NSA signals. A commonly used online questionnaire (Self-Administered Voice Rating questionnaire) was also implemented to track participants' perception of vocal fatigue.

RESULTS

The NSA wearables stayed in place for all participants despite active body movements during the acting. The ensued body noise did not interfere with the NSA signal quality. All planned acoustic metrics were successfully derived from NSA signals and their numerical values were comparable with literature data. For a 4-hour long voice acting, the averaged distance dose was about 8354 m with no gender differences. Participants perceived vocal fatigue as early as 2 hours after the start of voice acting, with recovery 24-48 hours after the acting session. Among all acoustic metrics across phonation tasks, cepstral peak prominence and spectral tilt from the passage reading most closely mirrored trends in perceived fatigue.

CONCLUSIONS

The ecological validity of an in-house NSA wearable was vetted in a workplace setting. One key application of this wearable is to prompt occupational voice users when their vocal safety limits are reached for duly protection. Signal processing algorithms can thus be further developed for near real-time estimation of clinically relevant metrics, such as accumulated distance dose, cepstral peak prominence, and spectral tilt. This functionality will enable continuous self-awareness of vocal behavior and protection of vocal safety in occupational voice users.

Extended High Frequency Hearing, but Not Tinnitus, Is Associated With Every-Day Cognitive Performance

Research into the potential associations between tinnitus and cognition has investigated specific cognitive domains in laboratory settings despite adults with tinnitus reporting broad cognitive difficulties in every-day life. To address this limitation, the present study compared performance and perceived exertion on a visual office-like task in 38 adults with tinnitus (19 with normal hearing and 19 with hearing loss) and 38 adults without tinnitus (19 with normal hearing and 19 with hearing loss) matched for age, sex and educational background. All participants were also assessed for hearing, anxiety and depression, and participants with tinnitus were also assessed for tinnitus handicap. No associations were found between presence of tinnitus and cognitive performance (mean total rate correct score on the visual office-like task being 2.9 for the tinnitus group, 2.8 for the control group, p = 0.612) and perceived exertion (mean ratings of perceived exertion on the Borg CR10-scale being 5.8 for the tinnitus group, 6.5 for the control group, p = 0.063) on the visual office-like task when corrected for standard (0.125 to 8 kHz) and extended high frequency (10 to 16 kHz) hearing thresholds, anxiety, and depression. The correction for extended high frequency average (10, 12.5, 14, and 16 kHz) hearing threshold was significant for performance (p = 0.009) but not perceived exertion on the visual office-like task. Overall, the results showed extended high frequency hearing, but not tinnitus, was associated with every-day cognitive performance. This indicates clinical testing of hearing thresholds above 8 kHz could support clinicians’ identification and management of cognitive difficulties. One management method suggested by the current findings would include provision of auditory stimulation at frequencies exceeding the frequency response of many current hearing aids.

Relationships Among Temporal Fine Structure Sensitivity, Transient Storage Capacity, and Ultra-High Frequency Hearing Thresholds in Tinnitus Patients and Normal Adults of Different Ages

Background

Elderlies and tinnitus patients often find it challenging to process acoustic signals in noisy environments. The sensitivity to temporal fine structure (TFS), the transient storage capacity for TFS, and the ultra-high frequency (UHF) thresholds are all associated with aging-related damage, evidenced by speech-in-noise perception deficits. In the present study, we aimed to investigate the relationships among TFS sensitivity, transient storage capacity, and UHF thresholds in tinnitus patients and normal adults of different ages.

Methods

In the present study, 38 tinnitus patients (age ranging from 21 to 65) and 23 non-tinnitus adults (age ranging from 22 to 56) were enrolled, and some of their auditory indicators were examined, including the TFS-adaptive frequency (TFS-AF), break in interaural correlation (BIAC) delay threshold, and UHF thresholds.

Results

We found no significant difference in TFS-AF thresholds and BIAC delay thresholds between the tinnitus group and normal group, while their relationships with age were more evident in the tinnitus group. Moreover, these two tests were only significantly correlated in the tinnitus group. UHF thresholds were significantly correlated with TFS-AF thresholds only in the tinnitus group, suggesting that the UHF hearing was positively associated with the TFS sensitivity.

Conclusion

These findings indicated that the influencing factors, such as tinnitus and UHF thresholds, should be fully considered when examining age-related hearing decline, because the combination of tinnitus and poor UHF hearing might play a role in affecting hearing ability, such as TFS sensitivity.

Extended high-frequency audiometry in research and clinical practice

Audiometric testing in research and in clinical settings rarely considers frequencies above 8 kHz. However, the sensitivity of young healthy ears extends to 20 kHz, and there is increasing evidence that testing in the extended high-frequency (EHF) region, above 8 kHz, might provide valuable additional information. Basal (EHF) cochlear regions are especially sensitive to the effects of aging, disease, ototoxic drugs, and possibly noise exposure. Hence, EHF loss may be an early warning of damage, useful for diagnosis and for monitoring hearing health. In certain environments, speech perception may rely on EHF information, and there is evidence for an association between EHF loss and speech perception difficulties, although this may not be causal: EHF loss may instead be a marker for sub-clinical damage at lower frequencies. If there is a causal relation, then amplification in the EHF range may be beneficial if the technical difficulties can be overcome. EHF audiometry in the clinic presents with no particular difficulty, the biggest obstacle being lack of specialist equipment. Currently, EHF audiometry has limited but increasing clinical application. With the development of international guidelines and standards, it is likely that EHF testing will become widespread in future.