Re-examining selective adaptation: Fatiguing feature detectors, or distributional learning?

What we do (not) know about the mechanisms underlying adaptive speech perception: A computational framework and review

Speech from unfamiliar talkers can be difficult to comprehend initially. These difficulties tend to dissipate with exposure, sometimes within minutes or less. Adaptivity in response to unfamiliar input is now considered a fundamental property of speech perception, and research over the past two decades has made substantial progress in identifying its characteristics. The mechanisms underlying adaptive speech perception, however, remain unknown. Past work has attributed facilitatory effects of exposure to any one of three qualitatively different hypothesized mechanisms: (1) low-level, pre-linguistic, signal normalization, (2) changes in/selection of linguistic representations, or (3) changes in post-perceptual decision-making. Direct comparisons of these hypotheses, or combinations thereof, have been lacking. We describe a general computational framework for adaptive speech perception (ASP) that-for the first time-implements all three mechanisms. We demonstrate how the framework can be used to derive predictions for experiments on perception from the acoustic properties of the stimuli. Using this approach, we find that-at the level of data analysis presently employed by most studies in the field-the signature results of influential experimental paradigms do not distinguish between the three mechanisms. This highlights the need for a change in research practices, so that future experiments provide more informative results. We recommend specific changes to experimental paradigms and data analysis. All data and code for this study are shared via OSF, including the R markdown document that this article is generated from, and an R library that implements the models we present.

Selective adaptation of German /r/: A role for perceptual saliency

In three experiments, we examined selective adaptation of German /r/ depending on the positional and allophonic overlap between adaptors and targets. A previous study had shown that selective adaptation effects with /r/ in Dutch require allophonic overlap between adaptor and target. We aimed at replicating this finding in German, which also has many allophones of /r/. German post-vocalic /r/ is often vocalized, and pre-vocalic /r/ can occur in at least three forms: uvular fricative [ʁ], uvular trill [ʀ] and alveolar trill [r]. We tested selective adaptation between these variants. The critical questions were whether an allophonic overlap is necessary for adaptation or whether phonemic overlap is sufficient to generate an adaptation effect. Surprisingly, our results show that both assertations are wrong: Adaptation does not require an allophonic overlap between adaptors and target and neither is phonemic overlap sufficient. Even more surprisingly, trilled adaptors led to more adaptation for a uvular-fricative target than uvular-fricative adaptors themselves. We suggest that the perceptual salience of the adaptors may be a hitherto underestimated influence on selective adaptation.

Speech-specific perceptual adaptation deficits in children and adults with dyslexia

According to several influential theoretical frameworks, phonological deficits in dyslexia result from reduced sensitivity to acoustic cues that are essential for the development of robust phonemic representations. Some accounts suggest that these deficits arise from impairments in rapid auditory adaptation processes that are either speech-specific or domain-general. Here, we examined the specificity of auditory adaptation deficits in dyslexia using a nonlinguistic tone anchoring (adaptation) task and a linguistic selective adaptation task in children and adults with and without dyslexia. Children and adults with dyslexia had elevated tone-frequency discrimination thresholds, but both groups benefited from anchoring to repeated stimuli to the same extent as typical readers. Additionally, although both dyslexia groups had overall reduced accuracy for speech sound identification, only the child group had reduced categorical perception for speech. Across both age groups, individuals with dyslexia had reduced perceptual adaptation to speech. These results highlight broad auditory perceptual deficits across development in individuals with dyslexia for both linguistic and nonlinguistic domains, but speech-specific adaptation deficits. Finally, mediation models in children and adults revealed that the causal pathways from basic perception and adaptation to phonological awareness through speech categorization were not significant. Thus, rather than having causal effects, perceptual deficits may co-occur with the phonological deficits in dyslexia across development. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Evidence For Selective Adaptation and Recalibration in the Perception of Lexical Stress

Individuals vary in how they produce speech. This variability affects both the segments (vowels and consonants) and the suprasegmental properties of their speech (prosody). Previous literature has demonstrated that listeners can adapt to variability in how different talkers pronounce the segments of speech. This study shows that listeners can also adapt to variability in how talkers produce lexical stress. Experiment 1 demonstrates a selective adaptation effect in lexical stress perception: repeatedly hearing Dutch trochaic words biased perception of a subsequent lexical stress continuum towards more iamb responses. Experiment 2 demonstrates a recalibration effect in lexical stress perception: when ambiguous suprasegmental cues to lexical stress were disambiguated by lexical orthographic context as signaling a trochaic word in an exposure phase, Dutch participants categorized a subsequent test continuum as more trochee-like. Moreover, the selective adaptation and recalibration effects generalized to novel words, not encountered during exposure. Together, the experiments demonstrate that listeners also flexibly adapt to variability in the suprasegmental properties of speech, thus expanding our understanding of the utility of listener adaptation in speech perception. Moreover, the combined outcomes speak for an architecture of spoken word recognition involving abstract prosodic representations at a prelexical level of analysis.

Repetitive Exposure to Orofacial Somatosensory Inputs in Speech Perceptual Training Modulates Vowel Categorization in Speech Perception

Orofacial somatosensory inputs may play a role in the link between speech perception and production. Given the fact that speech motor learning, which involves paired auditory and somatosensory inputs, results in changes to speech perceptual representations, somatosensory inputs may also be involved in learning or adaptive processes of speech perception. Here we show that repetitive pairing of somatosensory inputs and sounds, such as occurs during speech production and motor learning, can also induce a change of speech perception. We examined whether the category boundary between /ε/ and /a/ was changed as a result of perceptual training with orofacial somatosensory inputs. The experiment consisted of three phases: Baseline, Training, and Aftereffect. In all phases, a vowel identification test was used to identify the perceptual boundary between /ε/ and /a/. In the Baseline and the Aftereffect phase, an adaptive method based on the maximum-likelihood procedure was applied to detect the category boundary using a small number of trials. In the Training phase, we used the method of constant stimuli in order to expose participants to stimulus variants which covered the range between /ε/ and /a/ evenly. In this phase, to mimic the sensory input that accompanies speech production and learning in an experimental group, somatosensory stimulation was applied in the upward direction when the stimulus sound was presented. A control group (CTL) followed the same training procedure in the absence of somatosensory stimulation. When we compared category boundaries prior to and following paired auditory-somatosensory training, the boundary for participants in the experimental group reliably changed in the direction of /ε/, indicating that the participants perceived /a/ more than /ε/ as a consequence of training. In contrast, the CTL did not show any change. Although a limited number of participants were tested, the perceptual shift was reduced and almost eliminated 1 week later. Our data suggest that repetitive exposure of somatosensory inputs in a task that simulates the sensory pairing which occurs during speech production, changes perceptual system and supports the idea that somatosensory inputs play a role in speech perceptual adaptation, probably contributing to the formation of sound representations for speech perception.

Selective adaptation in sentence comprehension: Evidence from event-related brain potentials

In this study, two event-related potential experiments were conducted to investigate whether readers adapt their expectations to morphosyntactically (Experiment 1) or semantically (Experiment 2) anomalous sentences when they are repeatedly exposed to them. To address this issue, we experimentally manipulated the probability of occurrence of grammatical sentences and syntactically and semantically anomalous sentences through experiments. For the low probability block, anomalous sentences were presented less frequently than grammatical sentences (with a ratio of 1 to 4), while they were presented as frequently as grammatical sentences in the equal probability block. Experiment 1 revealed a smaller P600 effect for morphosyntactic violations in the equal probability block than in the low probability block. Linear mixed-effects models were used to examine how the size of the P600 effect changed as the experiment went along. The results showed that the smaller P600 effect of the equal probability block resulted from an amplitude's decline in morphosyntactically violated sentences over the course of the experiment, suggesting an adaptation to morphosyntactic violations. In Experiment 2, semantically anomalous sentences elicited a larger N400 effect than their semantically natural counterparts regardless of probability manipulation. Little evidence was found in favour of adaptation to semantic violations in that the processing cost associated with the N400 did not decrease over the course of the experiment. Therefore, a dynamic aspect of language-processing system was demonstrated in this study. We will discuss why the language-processing system shows a selective adaptation to morphosyntactic violations.

Talker-specific pronunciation or speech error? Discounting (or not) atypical pronunciations during speech perception

Perceptual recalibration allows listeners to adapt to talker-specific pronunciations, such as atypical realizations of specific sounds. Such recalibration can facilitate robust speech recognition. However, indiscriminate recalibration following any atypically pronounced words also risks interpreting pronunciations as characteristic of a talker that are in reality because of incidental, short-lived factors (such as a speech error). We investigate whether the mechanisms underlying perceptual recalibration involve inferences about the causes for unexpected pronunciations. In 5 experiments, we ask whether perceptual recalibration is blocked if the atypical pronunciations of an unfamiliar talker can also be attributed to other incidental causes. We investigated 3 type of incidental causes for atypical pronunciations: the talker is intoxicated, the talker speaks unusually fast, or the atypical pronunciations occur only in the context of tongue twisters. In all 5 experiments, we find robust evidence for perceptual recalibration, but little evidence that the presence of incidental causes block perceptual recalibration. We discuss these results in light of other recent findings that incidental causes can block perceptual recalibration. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

The Role of Lexical Status and Individual Differences for Perceptual Learning in Younger and Older Adults

PURPOSE

This study examined whether older adults remain perceptually flexible when presented with ambiguities in speech in the absence of lexically disambiguating information. We expected older adults to show less perceptual learning when top-down information was not available. We also investigated whether individual differences in executive function predicted perceptual learning in older and younger adults.

METHOD

Younger (n = 31) and older adults (n = 27) completed 2 perceptual learning tasks composed of a pretest, exposure, and posttest phase. Both learning tasks exposed participants to clear and ambiguous speech tokens, but crucially, the lexically guided learning task provided disambiguating lexical information whereas the distributional learning task did not. Participants also performed several cognitive tasks to investigate individual differences in working memory, vocabulary, and attention-switching control.

RESULTS

We found that perceptual learning is maintained in older adults, but that learning may be stronger in contexts where top-down information is available. Receptive vocabulary scores predicted learning across both age groups and in both learning tasks.

CONCLUSIONS

Implicit learning is maintained with age across different learning conditions but remains stronger when lexically biasing information is available. We find that receptive vocabulary is relevant for learning in both types of learning tasks, suggesting the importance of vocabulary knowledge for adapting to ambiguities in speech.

Structure in talker variability: How much is there and how much can it help?

One of the persistent puzzles in understanding human speech perception is how listeners cope with talker variability. One thing that might help listeners is structure in talker variability: rather than varying randomly, talkers of the same gender, dialect, age, etc. tend to produce language in similar ways. Listeners are sensitive to this covariation between linguistic variation and socio-indexical variables. In this paper I present new techniques based on ideal observer models to quantify (1) the amount and type of structure in talker variation (informativity of a grouping variable), and (2) how useful such structure can be for robust speech recognition in the face of talker variability (the utility of a grouping variable). I demonstrate these techniques in two phonetic domains-word-initial stop voicing and vowel identity-and show that these domains have different amounts and types of talker variability, consistent with previous, impressionistic findings. An R package (phondisttools) accompanies this paper, and the source and data are available from osf.io/zv6e3.

Mapping the Speech Code: Cortical Responses Linking the Perception and Production of Vowels

The acoustic realization of speech is constrained by the physical mechanisms by which it is produced. Yet for speech perception, the degree to which listeners utilize experience derived from speech production has long been debated. In the present study, we examined how sensorimotor adaptation during production may affect perception, and how this relationship may be reflected in early vs. late electrophysiological responses. Participants first performed a baseline speech production task, followed by a vowel categorization task during which EEG responses were recorded. In a subsequent speech production task, half the participants received shifted auditory feedback, leading most to alter their articulations. This was followed by a second, post-training vowel categorization task. We compared changes in vowel production to both behavioral and electrophysiological changes in vowel perception. No differences in phonetic categorization were observed between groups receiving altered or unaltered feedback. However, exploratory analyses revealed correlations between vocal motor behavior and phonetic categorization. EEG analyses revealed correlations between vocal motor behavior and cortical responses in both early and late time windows. These results suggest that participants' recent production behavior influenced subsequent vowel perception. We suggest that the change in perception can be best characterized as a mapping of acoustics onto articulation.

Dyslexics' faster decay of implicit memory for sounds and words is manifested in their shorter neural adaptation

Dyslexia is a prevalent reading disability whose underlying mechanisms are still disputed. We studied the neural mechanisms underlying dyslexia using a simple frequency-discrimination task. Though participants were asked to compare the two tones in each trial, implicit memory of previous trials affected their responses. We hypothesized that implicit memory decays faster among dyslexics. We tested this by increasing the temporal intervals between consecutive trials, and by measuring the behavioral impact and ERP responses from the auditory cortex. Dyslexics showed a faster decay of implicit memory effects on both measures, with similar time constants. Finally, faster decay of implicit memory also characterized the impact of sound regularities in benefitting dyslexics' oral reading rate. Their benefit decreased faster as a function of the time interval from the previous reading of the same non-word. We propose that dyslexics’ shorter neural adaptation paradoxically accounts for their longer reading times, since it reduces their temporal window of integration of past stimuli, resulting in noisier and less reliable predictions for both simple and complex stimuli. Less reliable predictions limit their acquisition of reading expertise.

DOI:http://dx.doi.org/10.7554/eLife.20557.001

The term “dyslexia” comes from the Greek for “difficulty with words”. People with dyslexia struggle with reading and spelling: they may mix up letters within words and tend to read and write more slowly than others. However, not every symptom of dyslexia is related to literacy. Affected individuals also differ from good readers on simple perceptual tasks, such as distinguishing between tones of different frequencies.

In a series of trials involving discrimination between pairs of tones, a person’s performance on each trial will be influenced by the tones presented on previous trials. Both good readers and individuals with dyslexia automatically form a subconscious memory of the tones they hear, and use this memory to guide their performance on subsequent trials. However, people with dyslexia benefit less from this effect than good readers.

Jaffe-Dax et al. have now identified the mechanism that underlies this phenomenon, revealing new insights into how dyslexia influences brain activity. By varying the interval between successive pairs of tones, the experiments showed that the memory of previous tones decays faster in people with dyslexia than in good readers. A similar effect occurs when the stimuli are nonsense words. Both good and poor readers manage to read nonsense words more quickly on their second attempt. However, people with dyslexia benefit less from the previous exposure when the gap between repetitions is longer than a couple of seconds.

Further studies are needed to determine whether and how the faster decay of memory traces for words is related to impaired reading ability in people with dyslexia. One possibility is that the faster decay of memory traces makes it more difficult to predict future stimuli, which may impair reading. An imaging study is underway to investigate where in the brain this rapid decay of memory traces occurs.

DOI:http://dx.doi.org/10.7554/eLife.20557.002

Incremental implicit learning of bundles of statistical patterns

Forming an accurate representation of a task environment often takes place incrementally as the information relevant to learning the representation only unfolds over time. This incremental nature of learning poses an important problem: it is usually unclear whether a sequence of stimuli consists of only a single pattern, or multiple patterns that are spliced together. In the former case, the learner can directly use each observed stimulus to continuously revise its representation of the task environment. In the latter case, however, the learner must first parse the sequence of stimuli into different bundles, so as to not conflate the multiple patterns. We created a video-game statistical learning paradigm and investigated (1) whether learners without prior knowledge of the existence of multiple "stimulus bundles" - subsequences of stimuli that define locally coherent statistical patterns - could detect their presence in the input and (2) whether learners are capable of constructing a rich representation that encodes the various statistical patterns associated with bundles. By comparing human learning behavior to the predictions of three computational models, we find evidence that learners can handle both tasks successfully. In addition, we discuss the underlying reasons for why the learning of stimulus bundles occurs even when such behavior may seem irrational.

Talker-specificity and adaptation in quantifier interpretation

Linguistic meaning has long been recognized to be highly context-dependent. Quantifiers like many and some provide a particularly clear example of context-dependence. For example, the interpretation of quantifiers requires listeners to determine the relevant domain and scale. We focus on another type of context-dependence that quantifiers share with other lexical items: talker variability. Different talkers might use quantifiers with different interpretations in mind. We used a web-based crowdsourcing paradigm to study participants' expectations about the use of many and some based on recent exposure. We first established that the mapping of some and many onto quantities (candies in a bowl) is variable both within and between participants. We then examined whether and how listeners' expectations about quantifier use adapts with exposure to talkers who use quantifiers in different ways. The results demonstrate that listeners can adapt to talker-specific biases in both how often and with what intended meaning many and some are used.