1
|
Attentional control via synaptic gain mechanisms in auditory streaming. Brain Res 2021; 1778:147720. [PMID: 34785256 DOI: 10.1016/j.brainres.2021.147720] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/30/2021] [Revised: 09/13/2021] [Accepted: 11/05/2021] [Indexed: 11/21/2022]
Abstract
Attention is a crucial component in sound source segregation allowing auditory objects of interest to be both singled out and held in focus. Our study utilizes a fundamental paradigm for sound source segregation: a sequence of interleaved tones, A and B, of different frequencies that can be heard as a single integrated stream or segregated into two streams (auditory streaming paradigm). We focus on the irregular alternations between integrated and segregated that occur for long presentations, so-called auditory bistability. Psychaoustic experiments demonstrate how attentional control, a listener's intention to experience integrated or segregated, biases perception in favour of different perceptual interpretations. Our data show that this is achieved by prolonging the dominance times of the attended percept and, to a lesser extent, by curtailing the dominance times of the unattended percept, an effect that remains consistent across a range of values for the difference in frequency between A and B. An existing neuromechanistic model describes the neural dynamics of perceptual competition downstream of primary auditory cortex (A1). The model allows us to propose plausible neural mechanisms for attentional control, as linked to different attentional strategies, in a direct comparison with behavioural data. A mechanism based on a percept-specific input gain best accounts for the effects of attentional control.
Collapse
|
2
|
Wasiuk PA, Lavandier M, Buss E, Oleson J, Calandruccio L. The effect of fundamental frequency contour similarity on multi-talker listening in older and younger adults. THE JOURNAL OF THE ACOUSTICAL SOCIETY OF AMERICA 2020; 148:3527. [PMID: 33379934 PMCID: PMC7863686 DOI: 10.1121/10.0002661] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/04/2023]
Abstract
Older adults with hearing loss have greater difficulty recognizing target speech in multi-talker environments than young adults with normal hearing, especially when target and masker speech streams are perceptually similar. A difference in fundamental frequency (f0) contour depth is an effective stream segregation cue for young adults with normal hearing. This study examined whether older adults with varying degrees of sensorineural hearing loss are able to utilize differences in target/masker f0 contour depth to improve speech recognition in multi-talker listening. Speech recognition thresholds (SRTs) were measured for speech mixtures composed of target/masker streams with flat, normal, and exaggerated speaking styles, in which f0 contour depth systematically varied. Computational modeling estimated differences in energetic masking across listening conditions. Young adults had lower SRTs than older adults; a result that was partially explained by differences in audibility predicted by the model. However, audibility differences did not explain why young adults experienced a benefit from mismatched target/masker f0 contour depth, while in most conditions, older adults did not. Reduced ability to use segregation cues (differences in target/masker f0 contour depth), and deficits grouping speech with variable f0 contours likely contribute to difficulties experienced by older adults in challenging acoustic environments.
Collapse
Affiliation(s)
- Peter A Wasiuk
- Department of Psychological Sciences, 11635 Euclid Avenue, Case Western Reserve University, Cleveland, Ohio 44106, USA
| | - Mathieu Lavandier
- Univ. Lyon, ENTPE, Laboratoire Génie Civil et Bâtiment, Rue M. Audin, Vaulx-en-Velin Cedex, 69518, France
| | - Emily Buss
- Department of Otolaryngology/Head and Neck Surgery, University of North Carolina, CB#7070, Chapel Hill, North Carolina 27599, USA
| | - Jacob Oleson
- Department of Biostatistics, N300 CPHB, University of Iowa, 145 North Riverside Drive, Iowa City, Iowa 52242-2007, USA
| | - Lauren Calandruccio
- Department of Psychological Sciences, 11635 Euclid Avenue, Case Western Reserve University, Cleveland, Ohio 44106, USA
| |
Collapse
|
3
|
Gustafson SJ, Grose J, Buss E. Perceptual organization and stability of auditory streaming for pure tones and /ba/ stimuli. THE JOURNAL OF THE ACOUSTICAL SOCIETY OF AMERICA 2020; 148:EL159. [PMID: 32873027 PMCID: PMC7438158 DOI: 10.1121/10.0001744] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/26/2020] [Revised: 07/23/2020] [Accepted: 07/27/2020] [Indexed: 06/11/2023]
Abstract
The dynamics of auditory stream segregation were evaluated using repeating triplets composed of pure tones or the syllable /ba/. Stimuli differed in frequency (tones) or fundamental frequency (speech) by 4, 6, 8, or 10 semitones, and the standard frequency was either 250 Hz (tones and speech) or 400 Hz (tones). Twenty normal-hearing adults participated. For both tones and speech, a two-stream percept became more likely as frequency separation increased. Perceptual organization for speech tended to be more integrated and less stable compared to tones. Results suggest that prior data patterns observed with tones in this paradigm may generalize to speech stimuli.
Collapse
Affiliation(s)
- Samantha J Gustafson
- Department of Communication Sciences and Disorders, University of Utah, 390 South 1530 East, Salt Lake City, Utah 84112, USA
| | - John Grose
- Department of Otolaryngology-Head and Neck Surgery, University of North Carolina, 170, Manning Drive, Chapel Hill, North Carolina 27599, , ,
| | - Emily Buss
- Department of Otolaryngology-Head and Neck Surgery, University of North Carolina, 170, Manning Drive, Chapel Hill, North Carolina 27599, , ,
| |
Collapse
|
4
|
Baltzell LS, Cho AY, Swaminathan J, Best V. Spectro-temporal weighting of interaural time differences in speech. THE JOURNAL OF THE ACOUSTICAL SOCIETY OF AMERICA 2020; 147:3883. [PMID: 32611137 PMCID: PMC7297545 DOI: 10.1121/10.0001418] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/23/2019] [Revised: 05/06/2020] [Accepted: 05/18/2020] [Indexed: 05/19/2023]
Abstract
Numerous studies have demonstrated that the perceptual weighting of interaural time differences (ITDs) is non-uniform in time and frequency, leading to reports of spectral and temporal "dominance" regions. It is unclear however, how these dominance regions apply to spectro-temporally complex stimuli such as speech. The authors report spectro-temporal weighting functions for ITDs in a pair of naturally spoken speech tokens ("two" and "eight"). Each speech token was composed of two phonemes, and was partitioned into eight frequency regions over two time bins (one time bin for each phoneme). To derive lateralization weights, ITDs for each time-frequency bin were drawn independently from a normal distribution with a mean of 0 and a standard deviation of 200 μs, and listeners were asked to indicate whether the speech token was presented from the left or right. ITD thresholds were also obtained for each of the 16 time-frequency bins in isolation. The results suggest that spectral dominance regions apply to speech, and that ITDs carried by phonemes in the first position of the syllable contribute more strongly to lateralization judgments than ITDs carried by phonemes in the second position. The results also show that lateralization judgments are partially accounted for by ITD sensitivity across time-frequency bins.
Collapse
Affiliation(s)
- Lucas S Baltzell
- Department of Speech, Language, and Hearing Sciences, Boston University, 635 Commonwealth Avenue, Boston, Massachusetts 02215, USA
| | - Adrian Y Cho
- Department of Speech, Language, and Hearing Sciences, Boston University, 635 Commonwealth Avenue, Boston, Massachusetts 02215, USA
| | - Jayaganesh Swaminathan
- Department of Speech, Language, and Hearing Sciences, Boston University, 635 Commonwealth Avenue, Boston, Massachusetts 02215, USA
| | - Virginia Best
- Department of Speech, Language, and Hearing Sciences, Boston University, 635 Commonwealth Avenue, Boston, Massachusetts 02215, USA
| |
Collapse
|
5
|
Galle ME, Klein-Packard J, Schreiber K, McMurray B. What Are You Waiting For? Real-Time Integration of Cues for Fricatives Suggests Encapsulated Auditory Memory. Cogn Sci 2020; 43. [PMID: 30648798 DOI: 10.1111/cogs.12700] [Citation(s) in RCA: 12] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/03/2017] [Revised: 10/15/2018] [Accepted: 10/25/2018] [Indexed: 11/30/2022]
Abstract
Speech unfolds over time, and the cues for even a single phoneme are rarely available simultaneously. Consequently, to recognize a single phoneme, listeners must integrate material over several hundred milliseconds. Prior work contrasts two accounts: (a) a memory buffer account in which listeners accumulate auditory information in memory and only access higher level representations (i.e., lexical representations) when sufficient information has arrived; and (b) an immediate integration scheme in which lexical representations can be partially activated on the basis of early cues and then updated when more information arises. These studies have uniformly shown evidence for immediate integration for a variety of phonetic distinctions. We attempted to extend this to fricatives, a class of speech sounds which requires not only temporal integration of asynchronous cues (the frication, followed by the formant transitions 150-350 ms later), but also integration across different frequency bands and compensation for contextual factors like coarticulation. Eye movements in the visual world paradigm showed clear evidence for a memory buffer. Results were replicated in five experiments, ruling out methodological factors and tying the release of the buffer to the onset of the vowel. These findings support a general auditory account for speech by suggesting that the acoustic nature of particular speech sounds may have large effects on how they are processed. It also has major implications for theories of auditory and speech perception by raising the possibility of an encapsulated memory buffer in early auditory processing.
Collapse
Affiliation(s)
- Marcus E Galle
- Department of Psychological and Brain Sciences, University of Iowa
| | | | | | - Bob McMurray
- Department of Psychological and Brain Sciences, University of Iowa.,Department of Communication Sciences and Disorders, University of Iowa.,Department of Linguistics, University of Iowa.,Department of Otolaryngology, University of Iowa
| |
Collapse
|
6
|
Best V, Swaminathan J, Kopčo N, Roverud E, Shinn-Cunningham B. A "Buildup" of Speech Intelligibility in Listeners With Normal Hearing and Hearing Loss. Trends Hear 2019; 22:2331216518807519. [PMID: 30353783 PMCID: PMC6201174 DOI: 10.1177/2331216518807519] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022] Open
Abstract
The perception of simple auditory mixtures is known to evolve over time. For
instance, a common example of this is the “buildup” of stream segregation that
is observed for sequences of tones alternating in pitch. Yet very little is
known about how the perception of more complicated auditory scenes, such as
multitalker mixtures, changes over time. Previous data are consistent with the
idea that the ability to segregate a target talker from competing sounds
improves rapidly when stable cues are available, which leads to improvements in
speech intelligibility. This study examined the time course of this buildup in
listeners with normal and impaired hearing. Five simultaneous sequences of
digits, varying in length from three to six digits, were presented from five
locations in the horizontal plane. A synchronized visual cue at one location
indicated which sequence was the target on each trial. We observed a buildup in
digit identification performance, driven primarily by reductions in confusions
between the target and the maskers, that occurred over the course of three to
four digits. Performance tended to be poorer in listeners with hearing loss;
however, there was only weak evidence that the buildup was diminished or slowed
in this group.
Collapse
Affiliation(s)
- Virginia Best
- 1 Department of Speech, Language and Hearing Sciences, Boston University, MA, USA
| | | | - Norbert Kopčo
- 3 Faculty of Science, Institute of Computer Science, P. J. Safarik University, Kosice, Slovakia
| | - Elin Roverud
- 1 Department of Speech, Language and Hearing Sciences, Boston University, MA, USA
| | | |
Collapse
|
7
|
Wang D, Chen J. Supervised Speech Separation Based on Deep Learning: An Overview. IEEE/ACM TRANSACTIONS ON AUDIO, SPEECH, AND LANGUAGE PROCESSING 2018; 26:1702-1726. [PMID: 31223631 PMCID: PMC6586438 DOI: 10.1109/taslp.2018.2842159] [Citation(s) in RCA: 119] [Impact Index Per Article: 19.8] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/12/2023]
Abstract
Speech separation is the task of separating target speech from background interference. Traditionally, speech separation is studied as a signal processing problem. A more recent approach formulates speech separation as a supervised learning problem, where the discriminative patterns of speech, speakers, and background noise are learned from training data. Over the past decade, many supervised separation algorithms have been put forward. In particular, the recent introduction of deep learning to supervised speech separation has dramatically accelerated progress and boosted separation performance. This paper provides a comprehensive overview of the research on deep learning based supervised speech separation in the last several years. We first introduce the background of speech separation and the formulation of supervised separation. Then, we discuss three main components of supervised separation: learning machines, training targets, and acoustic features. Much of the overview is on separation algorithms where we review monaural methods, including speech enhancement (speech-nonspeech separation), speaker separation (multitalker separation), and speech dereverberation, as well as multimicrophone techniques. The important issue of generalization, unique to supervised learning, is discussed. This overview provides a historical perspective on how advances are made. In addition, we discuss a number of conceptual issues, including what constitutes the target source.
Collapse
Affiliation(s)
- DeLiang Wang
- Department of Computer Science and Engineering and the Center for Cognitive and Brain Sciences, The Ohio State University, Columbus, OH 43210 USA, and also with the Center of Intelligent Acoustics and Immersive Communications, Northwestern Polytechnical University, Xi'an 710072, China
| | - Jitong Chen
- Department of Computer Science and Engineering, The Ohio State University, Columbus, OH 43210 USA. He is now with Silicon Valley AI Lab, Baidu Research, Sunnyvale, CA 94089 USA
| |
Collapse
|
8
|
David M, Tausend AN, Strelcyk O, Oxenham AJ. Effect of age and hearing loss on auditory stream segregation of speech sounds. Hear Res 2018; 364:118-128. [PMID: 29602593 DOI: 10.1016/j.heares.2018.03.017] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/21/2017] [Revised: 02/09/2018] [Accepted: 03/15/2018] [Indexed: 10/17/2022]
Abstract
Segregating and understanding speech in complex environments is a major challenge for hearing-impaired (HI) listeners. It remains unclear to what extent these difficulties are dominated by direct interference, such as simultaneous masking, or by a failure of the mechanisms of stream segregation. This study compared older HI listeners' performance with that of young and older normal-hearing (NH) listeners in stream segregation tasks involving speech sounds. Listeners were presented with sequences of speech tokens, each consisting of a fricative consonant and a voiced vowel (CV). The CV tokens were concatenated into interleaved sequences that alternated in fundamental frequency (F0) and/or simulated vocal tract length (VTL). Each pair of interleaved sequences was preceded by a "word" consisting of two random tokens. The listeners were asked to indicate whether the word was present in the following interleaved sequences. The word, if present, occurred within one of the interleaved sequences, so that performance improved if the listeners were able to perceptually segregate the two sequences. Although HI listeners' identification of the speech tokens in isolation was poorer than that of the NH listeners, HI listeners were generally able to use both F0 and VTL cues to segregate the interleaved sequences. The results suggest that the difficulties experienced by HI listeners in complex acoustic environments cannot be explained by a loss of basic stream segregation abilities.
Collapse
Affiliation(s)
- Marion David
- Department of Psychology, University of Minnesota, Minnesota, 55455, USA.
| | - Alexis N Tausend
- Department of Psychology, University of Minnesota, Minnesota, 55455, USA
| | - Olaf Strelcyk
- Sonova U.S. Corporate Services, 4520 Weaver Parkway, Warrenville, IL, 60555, USA
| | - Andrew J Oxenham
- Department of Psychology, University of Minnesota, Minnesota, 55455, USA
| |
Collapse
|
9
|
Shearer DE, Molis MR, Bennett KO, Leek MR. Auditory stream segregation of iterated rippled noises by normal-hearing and hearing-impaired listeners. THE JOURNAL OF THE ACOUSTICAL SOCIETY OF AMERICA 2018; 143:378. [PMID: 29390743 PMCID: PMC5785299 DOI: 10.1121/1.5021333] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/16/2016] [Revised: 08/09/2017] [Accepted: 01/02/2018] [Indexed: 05/28/2023]
Abstract
Individuals with hearing loss are thought to be less sensitive to the often subtle variations of acoustic information that support auditory stream segregation. Perceptual segregation can be influenced by differences in both the spectral and temporal characteristics of interleaved stimuli. The purpose of this study was to determine what stimulus characteristics support sequential stream segregation by normal-hearing and hearing-impaired listeners. Iterated rippled noises (IRNs) were used to assess the effects of tonality, spectral resolvability, and hearing loss on the perception of auditory streams in two pitch regions, corresponding to 250 and 1000 Hz. Overall, listeners with hearing loss were significantly less likely to segregate alternating IRNs into two auditory streams than were normally hearing listeners. Low pitched IRNs were generally less likely to segregate into two streams than were higher pitched IRNs. High-pass filtering was a strong contributor to reduced segregation for both groups. The tonality, or pitch strength, of the IRNs had a significant effect on streaming, but the effect was similar for both groups of subjects. These data demonstrate that stream segregation is influenced by many factors including pitch differences, pitch region, spectral resolution, and degree of stimulus tonality, in addition to the loss of auditory sensitivity.
Collapse
Affiliation(s)
- Daniel E Shearer
- National Center for Rehabilitative Auditory Research, Portland VA Healthcare System. Portland, Oregon 97239, USA
| | - Michelle R Molis
- National Center for Rehabilitative Auditory Research, Portland VA Healthcare System. Portland, Oregon 97239, USA
| | - Keri O Bennett
- National Center for Rehabilitative Auditory Research, Portland VA Healthcare System. Portland, Oregon 97239, USA
| | - Marjorie R Leek
- National Center for Rehabilitative Auditory Research, Portland VA Healthcare System. Portland, Oregon 97239, USA
| |
Collapse
|
10
|
David M, Lavandier M, Grimault N, Oxenham AJ. Discrimination and streaming of speech sounds based on differences in interaural and spectral cues. THE JOURNAL OF THE ACOUSTICAL SOCIETY OF AMERICA 2017; 142:1674. [PMID: 28964066 PMCID: PMC5617732 DOI: 10.1121/1.5003809] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/09/2017] [Revised: 09/01/2017] [Accepted: 09/07/2017] [Indexed: 05/29/2023]
Abstract
Differences in spatial cues, including interaural time differences (ITDs), interaural level differences (ILDs) and spectral cues, can lead to stream segregation of alternating noise bursts. It is unknown how effective such cues are for streaming sounds with realistic spectro-temporal variations. In particular, it is not known whether the high-frequency spectral cues associated with elevation remain sufficiently robust under such conditions. To answer these questions, sequences of consonant-vowel tokens were generated and filtered by non-individualized head-related transfer functions to simulate the cues associated with different positions in the horizontal and median planes. A discrimination task showed that listeners could discriminate changes in interaural cues both when the stimulus remained constant and when it varied between presentations. However, discrimination of changes in spectral cues was much poorer in the presence of stimulus variability. A streaming task, based on the detection of repeated syllables in the presence of interfering syllables, revealed that listeners can use both interaural and spectral cues to segregate alternating syllable sequences, despite the large spectro-temporal differences between stimuli. However, only the full complement of spatial cues (ILDs, ITDs, and spectral cues) resulted in obligatory streaming in a task that encouraged listeners to integrate the tokens into a single stream.
Collapse
Affiliation(s)
- Marion David
- Department of Psychology, University of Minnesota, Minneapolis, Minnesota 55455, USA
| | - Mathieu Lavandier
- Univ Lyon, ENTPE, Laboratoire Génie Civil et bâtiment, Rue Maurice Audin, 69518 Vaulx-en-Velin Cedex, France
| | - Nicolas Grimault
- Centre de Recherche en Neurosciences de Lyon, Université Lyon 1, Cognition Auditive et Psychoacoustique, Avenue Tony Garnier, 69366 Lyon Cedex 07, France
| | - Andrew J Oxenham
- Department of Psychology, University of Minnesota, Minneapolis, Minnesota 55455, USA
| |
Collapse
|
11
|
Stachurski M, Summers RJ, Roberts B. Stream segregation of concurrent speech and the verbal transformation effect: Influence of fundamental frequency and lateralization cues. Hear Res 2017; 354:16-27. [PMID: 28843209 DOI: 10.1016/j.heares.2017.07.016] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/04/2017] [Revised: 07/25/2017] [Accepted: 07/31/2017] [Indexed: 10/19/2022]
Abstract
Repeating a recorded word produces verbal transformations (VTs); perceptual regrouping of acoustic-phonetic elements may contribute to this effect. The influence of fundamental frequency (F0) and lateralization grouping cues was explored by presenting two concurrent sequences of the same word resynthesized on different F0s (100 and 178 Hz). In experiment 1, listeners monitored both sequences simultaneously, reporting for each any change in stimulus identity. Three lateralization conditions were used - diotic, ±680-μs interaural time difference, and dichotic. Results were similar for the first two conditions, but fewer forms and later initial transformations were reported in the dichotic condition. This suggests that large lateralization differences per se have little effect - rather, there are more possibilities for regrouping when each ear receives both sequences. In the dichotic condition, VTs reported for one sequence were also more independent of those reported for the other. Experiment 2 used diotic stimuli and explored the effect of the number of sequences presented and monitored. The most forms and earliest transformations were reported when two sequences were presented but only one was monitored, indicating that high task demands decreased reporting of VTs for concurrent sequences. Overall, these findings support the idea that perceptual regrouping contributes to the VT effect.
Collapse
Affiliation(s)
- Marcin Stachurski
- Psychology, School of Life and Health Sciences, Aston University, Birmingham, B4 7ET, UK
| | - Robert J Summers
- Psychology, School of Life and Health Sciences, Aston University, Birmingham, B4 7ET, UK
| | - Brian Roberts
- Psychology, School of Life and Health Sciences, Aston University, Birmingham, B4 7ET, UK.
| |
Collapse
|