Implicit statistical learning in language processing: word predictability is the key

REHABILITATING LANGUAGE DISORDERS BY IMPROVING SEQUENTIAL PROCESSING: A REVIEW

Recent research suggests that language processing (LP) may rely heavily on sequential processing (SP), a cognitive ability that allows people to process the patterns of environmental stimuli that unfold in time, such as spoken language or music. Indeed, spoken language corresponds to a set of linguistic units (e.g., phonemes, syllables, words) that are organized in time in a non-random way, according to phonotactic and syntactic rules. In this review, we discuss recent research highlighting the importance of SP for learning and processing such linguistic regularities and argue that interventions focused on improving SP may provide a potentially effective way to rehabilitate language impairments. The first part of this review presents a series of findings showing that LP is intimately related to SP. We review the literature on populations with normal LP performance suggesting that LP relies upon SP. We then report two recent studies from our lab that demonstrated a direct link between LP and SP: (1) a behavioral study showing that variations on a non-linguistic SP task are significantly associated with LP, and (2) an event-related potential study showing that the neural correlates of SP interact with LP abilities in healthy adults. The second part of this review summarizes the literature suggesting that populations with LP impairments (such as language delays due to hearing loss, dyslexia, specific language impairment, and aphasia) also display SP impairments. Thus, disturbances to SP appear to be a commonality among what appears to be very different types of LP impairments, suggesting that impaired SP causes or exacerbates LP impairment. This leads to the third part of this review, where we first summarize recent findings from brain plasticity showing that: (1) cognitive training can improve cognitive processing, and that (2) increasing cognitive processing performance through training can result in a cognitive "transfer" by also increasing performance on other related cognitive skills. We then present a potentially new method for LP remediation that is based on the idea that some LP impairments might stem directly from SP disturbances and that improving SP processing will, via transfer, result in increased LP performance. This method was applied by our research team to conduct a study aimed at improving SP and LP mechanisms. To our knowledge, the SP training study presented here shows the first evidence that SP performance can be improved and therefore has strong clinical implications as a potentially effective and novel intervention to treat LP impairments.

Development, reliability, and validity of PRESTO: a new high-variability sentence recognition test

BACKGROUND

There is a pressing need for new clinically feasible speech recognition tests that are theoretically motivated, sensitive to individual differences, and access the core perceptual and neurocognitive processes used in speech perception. PRESTO (Perceptually Robust English Sentence Test Open-set) is a new high-variability sentence test designed to reflect current theories of exemplar-based learning, attention, and perception, including lexical organization and automatic encoding of indexical attributes. Using sentences selected from the TIMIT (Texas Instruments/Massachusetts Institute of Technology) speech corpus, PRESTO was developed to include talker and dialect variability. The test consists of lists balanced for talker gender, keywords, frequency, and familiarity.

PURPOSE

To investigate the performance, reliability, and validity of PRESTO.

RESEARCH DESIGN

In Phase I, PRESTO sentences were presented in multitalker babble at four signal-to-noise ratios (SNRs) to obtain a distribution of performance. In Phase II, participants returned and were tested on new PRESTO sentences and on HINT (Hearing In Noise Test) sentences presented in multitalker babble.

STUDY SAMPLE

Young, normal-hearing adults (N = 121) were recruited from the Indiana University community for Phase I. Participants who scored within the upper and lower quartiles of performance in Phase I were asked to return for Phase II (N = 40).

DATA COLLECTION AND ANALYSIS

In both Phase I and Phase II, participants listened to sentences presented diotically through headphones while seated in enclosed carrels at the Speech Research Laboratory at Indiana University. They were instructed to type in the sentence that they heard using keyboards interfaced to a computer. Scoring for keywords was completed offline following data collection. Phase I data were analyzed by determining the distribution of performance on PRESTO at each SNR and at the average performance across all SNRs. PRESTO reliability was analyzed by a correlational analysis of participant performance at test (Phase I) and retest (Phase II). PRESTO validity was analyzed by a correlational analysis of participant performance on PRESTO and HINT sentences tested in Phase II, and by an analysis of variance of within-subject factors of sentence test and SNR, and a between-subjects factor of group, based on level of Phase I performance.

RESULTS

A wide range of performance on PRESTO was observed; averaged across all SNRs, keyword accuracy ranged from 40.26 to 76.18% correct. PRESTO accuracy at retest (Phase II) was highly correlated with Phase I accuracy (r = 0.92, p < 0.001). PRESTO scores were also correlated with scores on HINT sentences (r = 0.52, p < 0.001). Phase II results showed an interaction between sentence test type and SNR [F(3, 114) = 121.36, p < 0.001], with better performance on HINT sentences at more favorable SNRs and better performance on PRESTO sentences at poorer SNRs.

CONCLUSIONS

PRESTO demonstrated excellent test/retest reliability. Although a moderate correlation was observed between PRESTO and HINT sentences, a different pattern of results occurred with the two types of sentences depending on the level of the competition, suggesting the use of different processing strategies. Findings from this study demonstrate the importance of high-variability materials for assessing and understanding individual differences in speech perception.

Language experience changes subsequent learning

What are the effects of experience on subsequent learning? We explored the effects of language-specific word order knowledge on the acquisition of sequential conditional information. Korean and English adults were engaged in a sequence learning task involving three different sets of stimuli: auditory linguistic (nonsense syllables), visual non-linguistic (nonsense shapes), and auditory non-linguistic (pure tones). The forward and backward probabilities between adjacent elements generated two equally probable and orthogonal perceptual parses of the elements, such that any significant preference at test must be due to either general cognitive biases, or prior language-induced biases. We found that language modulated parsing preferences with the linguistic stimuli only. Intriguingly, these preferences are congruent with the dominant word order patterns of each language, as corroborated by corpus analyses, and are driven by probabilistic preferences. Furthermore, although the Korean individuals had received extensive formal explicit training in English and lived in an English-speaking environment, they exhibited statistical learning biases congruent with their native language. Our findings suggest that mechanisms of statistical sequential learning are implicated in language across the lifespan, and experience with language may affect cognitive processes and later learning.

How hierarchical is language use?

It is generally assumed that hierarchical phrase structure plays a central role in human language. However, considerations of simplicity and evolutionary continuity suggest that hierarchical structure should not be invoked too hastily. Indeed, recent neurophysiological, behavioural and computational studies show that sequential sentence structure has considerable explanatory power and that hierarchical processing is often not involved. In this paper, we review evidence from the recent literature supporting the hypothesis that sequential structure may be fundamental to the comprehension, production and acquisition of human language. Moreover, we provide a preliminary sketch outlining a non-hierarchical model of language use and discuss its implications and testable predictions. If linguistic phenomena can be explained by sequential rather than hierarchical structure, this will have considerable impact in a wide range of fields, such as linguistics, ethology, cognitive neuroscience, psychology and computer science.

Advancing Our Understanding of the Link between Statistical Learning and Language Acquisition: The Need for Longitudinal Data

Mastery of language can be a struggle for some children. Amongst those that succeed in achieving this feat there is variability in proficiency. Cognitive scientists remain intrigued by this variation. A now substantial body of research suggests that language acquisition is underpinned by a child's capacity for statistical learning (SL). Moreover, a growing body of research has demonstrated that variability in SL is associated with variability in language proficiency. Yet, there is a striking lack of longitudinal data. To date, there has been no comprehensive investigation of whether a capacity for SL in young children is, in fact, associated with language proficiency in subsequent years. Here we review key studies that have led to the need for this longitudinal research. Advancing the language acquisition debate via longitudinal research has the potential to transform our understanding of typical development as well as disorders such as autism, specific language impairment, and dyslexia.

Visual Sequence Learning in Infancy: Domain-General and Domain-Specific Associations with Language

Research suggests that non-linguistic sequence learning abilities are an important contributor to language development (Conway, Bauernschmidt, Huang, & Pisoni, 2010). The current study investigated visual sequence learning as a possible predictor of vocabulary development in infants. Fifty-eight 8.5-month-old infants were presented with a three-location spatiotemporal sequence of multi-colored geometric shapes. Early language skills were assessed using the MacArthur-Bates CDI. Analyses of children's reaction times to the stimuli suggest that the extent to which infants demonstrated learning was significantly correlated with their vocabulary comprehension at the time of test and with their gestural comprehension abilities 5 months later. These findings suggest that visual sequence learning may have both domain-general and domain-specific associations with language learning.

Cross-Modal Distraction by Deviance

Unexpected task-irrelevant changes in the auditory or visual sensory channels have been shown to capture attention in an ineluctable manner and distract participants away from ongoing auditory or visual categorization tasks. We extend the study of this phenomenon by reporting the first within-participant comparison of deviance distraction in the tactile and auditory modalities. Using vibro-tactile-visual and auditory-visual cross-modal oddball tasks, we found that unexpected changes in the tactile and auditory modalities produced a number of functional similarities: A negative impact of distracter deviance on performance in the ongoing visual task, distraction on the subsequent trial (post-deviance distraction), and a similar decrease – but not the disappearance – of these effects across blocks. Despite these functional similarities, deviance distraction only correlated between the auditory and tactile modalities for the accuracy-based measure of deviance distraction and not for response latencies. Post-deviance distraction showed no correlation between modalities. Overall, the results suggest that behavioral deviance distraction may be underpinned by both modality-specific and multimodal mechanisms, while post-deviance distraction may predominantly relate to modality-specific processes.

Similar Neural Correlates for Language and Sequential Learning: Evidence from Event-Related Brain Potentials

We used event-related potentials (ERPs) to investigate the time course and distribution of brain activity while adults performed (a) a sequential learning task involving complex structured sequences, and (b) a language processing task. The same positive ERP deflection, the P600 effect, typically linked to difficult or ungrammatical syntactic processing, was found for structural incongruencies in both sequential learning as well as natural language, and with similar topographical distributions. Additionally, a left anterior negativity (LAN) was observed for language but not for sequential learning. These results are interpreted as an indication that the P600 provides an index of violations and the cost of integration of expectations for upcoming material when processing complex sequential structure. We conclude that the same neural mechanisms may be recruited for both syntactic processing of linguistic stimuli and sequential learning of structured sequence patterns more generally.

Structural priming as implicit learning: cumulative priming effects and individual differences

We explored the claim that structural priming is a case of implicit learning within the language production system. The experiment began with a baseline phase, in which we assessed participants' rates of production for double object and prepositional object constructions. Then participants were biased toward the production of either the double object or prepositional object construction. Finally, we again assessed participants' rates of production for the target constructions. Consistent with claims that structural priming is a case of implicit learning, we found that biasing participants toward the prepositional object construction produced stronger cumulative priming effects than did biasing participants toward the double object construction. We also found that individual differences in implicit learning were marginally correlated with overall rates of production for the double object construction. Participants who scored better on the learning task tended to produce fewer double object constructions.

Isolated words enhance statistical language learning in infancy

Infants are adept at tracking statistical regularities to identify word boundaries in pause-free speech. However, researchers have questioned the relevance of statistical learning mechanisms to language acquisition, since previous studies have used simplified artificial languages that ignore the variability of real language input. The experiments reported here embraced a key dimension of variability in infant-directed speech. English-learning infants (8-10 months) listened briefly to natural Italian speech that contained either fluent speech only or a combination of fluent speech and single-word utterances. Listening times revealed successful learning of the statistical properties of target words only when words appeared both in fluent speech and in isolation; brief exposure to fluent speech alone was not sufficient to facilitate detection of the words' statistical properties. This investigation suggests that statistical learning mechanisms actually benefit from variability in utterance length, and provides the first evidence that isolated words and longer utterances act in concert to support infant word segmentation.

Implicit memory in music and language

Research on music and language in recent decades has focused on their overlapping neurophysiological, perceptual, and cognitive underpinnings, ranging from the mechanism for encoding basic auditory cues to the mechanism for detecting violations in phrase structure. These overlaps have most often been identified in musicians with musical knowledge that was acquired explicitly, through formal training. In this paper, we review independent bodies of work in music and language that suggest an important role for implicitly acquired knowledge, implicit memory, and their associated neural structures in the acquisition of linguistic or musical grammar. These findings motivate potential new work that examines music and language comparatively in the context of the implicit memory system.

Implicit sequence learning in deaf children with cochlear implants

Deaf children with cochlear implants (CIs) represent an intriguing opportunity to study neurocognitive plasticity and reorganization when sound is introduced following a period of auditory deprivation early in development. Although it is common to consider deafness as affecting hearing alone, it may be the case that auditory deprivation leads to more global changes in neurocognitive function. In this paper, we investigate implicit sequence learning abilities in deaf children with CIs using a novel task that measured learning through improvement to immediate serial recall for statistically consistent visual sequences. The results demonstrated two key findings. First, the deaf children with CIs showed disturbances in their visual sequence learning abilities relative to the typically developing normal-hearing children. Second, sequence learning was significantly correlated with a standardized measure of language outcome in the CI children. These findings suggest that a period of auditory deprivation has secondary effects related to general sequencing deficits, and that disturbances in sequence learning may at least partially explain why some deaf children still struggle with language following cochlear implantation.

Insensitivity of the human sentence-processing system to hierarchical structure

Although it is generally accepted that hierarchical phrase structures are instrumental in describing human language, their role in cognitive processing is still debated. We investigated the role of hierarchical structure in sentence processing by implementing a range of probabilistic language models, some of which depended on hierarchical structure, and others of which relied on sequential structure only. All models estimated the occurrence probabilities of syntactic categories in sentences for which reading-time data were available. Relating the models' probability estimates to the data showed that the hierarchical-structure models did not account for variance in reading times over and above the amount of variance accounted for by all of the sequential-structure models. This suggests that a sentence's hierarchical structure, unlike many other sources of information, does not noticeably affect the generation of expectations about upcoming words.

Interference between sentence processing and probabilistic implicit sequence learning

Background

During sentence processing we decode the sequential combination of words, phrases or sentences according to previously learned rules. The computational mechanisms and neural correlates of these rules are still much debated. Other key issue is whether sentence processing solely relies on language-specific mechanisms or is it also governed by domain-general principles.

Methodology/Principal Findings

In the present study, we investigated the relationship between sentence processing and implicit sequence learning in a dual-task paradigm in which the primary task was a non-linguistic task (Alternating Serial Reaction Time Task for measuring probabilistic implicit sequence learning), while the secondary task were a sentence comprehension task relying on syntactic processing. We used two control conditions: a non-linguistic one (math condition) and a linguistic task (word processing task). Here we show that the sentence processing interfered with the probabilistic implicit sequence learning task, while the other two tasks did not produce a similar effect.

Conclusions/Significance

Our findings suggest that operations during sentence processing utilize resources underlying non-domain-specific probabilistic procedural learning. Furthermore, it provides a bridge between two competitive frameworks of language processing. It appears that procedural and statistical models of language are not mutually exclusive, particularly for sentence processing. These results show that the implicit procedural system is engaged in sentence processing, but on a mechanism level, language might still be based on statistical computations.

Why are auditory novels distracting? Contrasting the roles of novelty, violation of expectation and stimulus change

Past studies show that novel auditory stimuli, presented in the context of an otherwise repeated sound, capture participants' attention away from a focal task, resulting in measurable behavioral distraction. Novel sounds are traditionally defined as rare and unexpected but past studies have not sought to disentangle these concepts directly. Using a cross-modal oddball task, we contrasted these aspects orthogonally by manipulating the base rate and conditional probabilities of sound events. We report for the first time that behavioral distraction does not result from a sound's novelty per se but from the violation of the cognitive system's expectation based on the learning of conditional probabilities and, to some extent, the occurrence of a perceptual change from one sound to another.

Implicit learning of fifth- and sixth-order sequential probabilities

Serial reaction time (SRT) task studies have established that people can implicitly learn sequential contingencies as complex as fourth-order probabilities. The present study examined people's ability to learn fifth-order (Experiment 1) and sixth-order (Experiment 2) probabilities. Remarkably, people learned fifth- and sixth-order probabilities. This suggests that the implicit sequence learning mechanism can operate over a range of at least seven sequence elements.

Neuroanatomical characteristics and speech perception in noise in older adults

OBJECTIVES

Previous research has attributed older adults' difficulty with perceiving speech in noise to peripheral hearing loss. However, recent studies have suggested a more complex picture and implicate the central nervous system in sensation and sensory deficits. This study examines the relationship between the neuroanatomical structure of cognitive brain regions and the ability to perceive speech in noise in older adults. In particular, the neuroanatomical characteristics of the left ventral and dorsal prefrontal cortex (PFC) are considered relative to standard measures of hearing in noise.

DESIGN

The participants were 15 older and 14 younger right-handed native speakers of American English who had no neurologic deficits and scored better than normal on standardized cognitive tests. We measured the participants' peripheral hearing ability and their ability to perceive speech in noise using established tests. Anatomical magnetic resonance images were taken and analyzed to extract regional volumes and thicknesses of several key neuroanatomical structures.

RESULTS

Younger adults had better hearing sensitivity and better speech perception in noise ability than older adults. For the older adults only, the volume of the left pars triangularis and the cortical thickness of the left superior frontal gyrus were significant predictors of performance on the speech in noise test.

DISCUSSION

These findings suggest that, in addition to peripheral structures, the central nervous system also contributes to the ability to perceive speech in noise. In older adults, a decline in the relative volume and cortical thickness of the PFC during aging can therefore be a factor in a declining ability to perceive speech in a naturalistic environment. These findings are consistent with the decline-compensation hypothesis, which states that a decline in sensory processing caused by cognitive aging can be accompanied by an increase in the recruitment of more general cognitive areas as a means of compensation. We found that a larger PFC volume may compensate for declining peripheral hearing. Clinically, recognizing the contribution of the cerebral cortex expands treatment possibilities for hearing loss in older adults beyond peripheral hearing aids to include strategies for improving cognitive function. We conclude by considering several mechanisms by which the PFC may facilitate speech perception in noise, including inhibitory control, attention, cross-modal compensation, word prediction and phonological working memory, although no definitive conclusion can be drawn.

On-line individual differences in statistical learning predict language processing

Considerable individual differences in language ability exist among normally developing children and adults. Whereas past research have attributed such differences to variations in verbal working memory or experience with language, we test the hypothesis that individual differences in statistical learning may be associated with differential language performance. We employ a novel paradigm for studying statistical learning on-line, combining a serial-reaction time task with artificial grammar learning. This task offers insights into both the timecourse of and individual differences in statistical learning. Experiment 1 charts the micro-level trajectory for statistical learning of nonadjacent dependencies and provides an on-line index of individual differences therein. In Experiment 2, these differences are then shown to predict variations in participants' on-line processing of long-distance dependencies involving center-embedded relative clauses. The findings suggest that individual differences in the ability to learn from experience through statistical learning may contribute to variations in linguistic performance.

Procedural learning and dyslexia

Three major 'neural systems', specialized for different types of information processing, are the sensory, declarative, and procedural systems. It has been proposed (Trends Neurosci., 30(4), 135-141) that dyslexia may be attributable to impaired function in the procedural system together with intact declarative function. We provide a brief overview of the increasing evidence relating to the hypothesis, noting that the framework involves two main claims: first that 'neural systems' provides a productive level of description avoiding the underspecificity of cognitive descriptions and the overspecificity of brain structural accounts; and second that a distinctive feature of procedural learning is its extended time course, covering from minutes to months. In this article, we focus on the second claim. Three studies-speeded single word reading, long-term response learning, and overnight skill consolidation-are reviewed which together provide clear evidence of difficulties in procedural learning for individuals with dyslexia, even when the tasks are outside the literacy domain. The educational implications of the results are then discussed, and in particular the potential difficulties that impaired overnight procedural consolidation would entail. It is proposed that response to intervention could be better predicted if diagnostic tests on the different forms of learning were first undertaken.