Does complexity matter? Meta-analysis of learner performance in artificial grammar tasks

Visual artificial grammar learning across 1 year in 7-year-olds and adults

Acquiring sequential information is of utmost importance, for example, for language acquisition in children. Yet, the long-term storage of statistical learning in children is poorly understood. To address this question, 27 7-year-olds and 28 young adults completed four sessions of visual sequence learning (Year 1). From this sample, 16 7-year-olds and 20 young adults participated in another four equivalent sessions after a 12-month-delay (Year 2). The first three sessions of each year used Stimulus Set 1, and the last session used Stimulus Set 2 to investigate transfer effects. Each session consisted of alternating learning and test phases in a modified artificial grammar learning task. In Year 1, 7-year-olds and adults learned the regularities and showed transfer to Stimulus Set 2. Both groups retained their final performance level over the 1-year period. In Year 2, children and adults continued to improve with Stimulus Set 1 but did not show additional transfer gains. Adults overall outperformed children, but transfer effects were indistinguishable between both groups. The current results suggest that long-term memory traces are formed from repeated sequence learning that can be used to generalize sequence rules to new visual input. However, the current study did not provide evidence for a childhood advantage in learning and remembering sequence rules.

Languages with more speakers tend to be harder to (machine-)learn

Computational language models (LMs), most notably exemplified by the widespread success of OpenAI's ChatGPT chatbot, show impressive performance on a wide range of linguistic tasks, thus providing cognitive science and linguistics with a computational working model to empirically study different aspects of human language. Here, we use LMs to test the hypothesis that languages with more speakers tend to be easier to learn. In two experiments, we train several LMs-ranging from very simple n-gram models to state-of-the-art deep neural networks-on written cross-linguistic corpus data covering 1293 different languages and statistically estimate learning difficulty. Using a variety of quantitative methods and machine learning techniques to account for phylogenetic relatedness and geographical proximity of languages, we show that there is robust evidence for a relationship between learning difficulty and speaker population size. However, contrary to expectations derived from previous research, our results suggest that languages with more speakers tend to be harder to learn.

Can adults with developmental dyslexia apply statistical knowledge to a new context?

We investigated transfer of artificial grammar learning in adults with and without dyslexia in 3 experiments. In Experiment 1, participants implicitly learned an artificial grammar system and were tested on new items that included the same symbols. In Experiment 2, participants were given practice with letter strings and then tested on strings created with a different letter set. In Experiment 3, participants were given practice with shapes and then tested on strings created with different shapes. Results show that in Experiment 1, both groups demonstrated utilization of pre-trained instances in the subsequent grammaticality judgement task, while in Experiments 2 (orthographic) and 3 (nonorthographic), only typically developed participants demonstrated application of knowledge from training to test. A post hoc analysis comparing between the experiments suggests that being trained and tested on an orthographic task leads to better performance than a nonorthographic task among typically developed adults but not among adults with dyslexia. Taken together, it appears that following extensive training, individuals with dyslexia are able to form stable representations from sequential stimuli and use them in a subsequent task that utilizes strings of similar symbols. However, the manipulation of the symbols challenges this ability.

Modality, presentation, domain and training effects in statistical learning

While several studies suggest that the nature and properties of the input have significant effects on statistical learning, they have rarely been investigated systematically. In order to understand how input characteristics and their interactions impact statistical learning, we explored the effects of modality (auditory vs. visual), presentation type (serial vs. simultaneous), domain (linguistic vs. non-linguistic), and training type (random, starting small, starting big) on artificial grammar learning in young adults (N = 360). With serial presentation of stimuli, learning was more effective in the auditory than in the visual modality. However, with simultaneous presentation of visual and serial presentation of auditory stimuli, the modality effect was not present. We found a significant domain effect as well: a linguistic advantage over nonlinguistic material, which was driven by the domain effect in the auditory modality. Overall, the auditory linguistic condition had an advantage over other modality-domain types. Training types did not have any overall effect on learning; starting big enhanced performance only in the case of serial visual presentation. These results show that input characteristics such as modality, presentation type, domain and training type influence statistical learning, and suggest that their effects are also dependent on the specific stimuli and structure to be learned.

Implicit spatial sequential learning facilitates attentional selection in covert visual search. An event-related potentials study

Introduction

While most studies on implicit sequential learning focus on object learning, the hidden structure of target location and onset time can also be a subject of implicitly gathered knowledge. In our study, we wanted to investigate the effect of implicitly learned spatial and temporal sequential predictability on performance in a localization task in a paradigm in which covert selective attention is engaged. We were also interested in the neural mechanism of the facilitating effect of the predictable spatio-temporal context on visual search processes. Specifically, with the use of an event-related potential technique, we wanted to verify whether perceptual, attentional, and motor processes can be enhanced by the predictive spatio-temporal context of visual stimuli.

Methods

We analyzed data from 15 young, healthy adults who took part in an experimental electroencephalographic (EEG) study and performed a visual search localization task. Predictable sequences of four target locations and/or target onset times were presented in separate blocks of trials that formed the Space, Space- Time, and Time conditions. One block of trials with randomly presented stimuli served as a control condition.

Results

The behavioral results revealed that participants successfully learned only the spatial dimension of target predictability. Although spatial predictability was a response-relevant dimension, we found that attentional selection–instead of motor preparation–was the facilitation mechanism in this type of visual search task. This was manifested by a shorter latency and more negative amplitude of the N2pc component and the lack of an effect on the sLRP component. We observed no effect of predictability on perceptual processing (P1 component).

Discussion

We discuss these results with reference to the current knowledge on sequential learning. Our findings also contribute to the current debate on the predictive coding theory.

Unraveling the Interconnections Between Statistical Learning and Dyslexia: A Review of Recent Empirical Studies

One important aspect of human cognition involves the learning of structured information encountered in our environment, a phenomenon known as statistical learning. A growing body of research suggests that learning to read print is partially guided by learning the statistical contingencies existing between the letters within a word, and also between the letters and sounds to which the letters refer. Research also suggests that impairments to statistical learning ability may at least partially explain the difficulties experienced by individuals diagnosed with dyslexia. However, the findings regarding impaired learning are not consistent, perhaps partly due to the varied use of methodologies across studies - such as differences in the learning paradigms, stimuli used, and the way that learning is assessed - as well as differences in participant samples such as age and extent of the learning disorder. In this review, we attempt to examine the purported link between statistical learning and dyslexia by assessing a set of the most recent and relevant studies in both adults and children. Based on this review, we conclude that although there is some evidence for a statistical learning impairment in adults with dyslexia, the evidence for an impairment in children is much weaker. We discuss several suggestive trends that emerge from our examination of the research, such as issues related to task heterogeneity, possible age effects, the role of publication bias, and other suggestions for future research such as the use of neural measures and a need to better understand how statistical learning changes across typical development. We conclude that no current theoretical framework of dyslexia fully captures the extant research findings on statistical learning.

Artificial grammar learning is facilitated by distributed practice: Evidence from a letter reordering task

Previous studies have shown that distributed practice-a training strategy that is known to facilitate memory-is likely to result in greater learning than massed practice. This effect has been demonstrated largely in explicit tasks. The purpose of this study was to test whether statistical learning of artificial grammar is affected by the lag between learning sessions overall, and by high and low complexity stimuli (as measure by chunk strength). Two groups (spaced-short and spaced-long) learned strings of letters created according to a set of rules and were required to produce new strings using given letter sets. For the spaced-short group, the two learning sessions, each including training and a test phase, took place sequentially with a 10-min break, whereas for the spaced-long group, learning sessions were distributed across two days (1-day lag). Overall results showed improved performance following spaced-long practice compared to spaced-short practice. The results also indicated that in the low chunk strength strings (indicating high complexity), both groups demonstrated similar improvement from first to second testing, while in the high chunk strength strings (indicating low complexity), improvement in letter reordering performance was significantly higher when the learning sessions were distributed across two days. This pattern of findings suggests that stimuli complexity affects the extent to which distributed practice enhance artificial grammar learning.

Stimulus variation-based training enhances artificial grammar learning

The current study was designed to explore whether statistical learning ability is affected by the diversity of the stimulus set used in the training phase. The effect of stimulus diversity was assessed by controlling and manipulating the number of exposures to a given set and the number of unique strings presented to the learner during the training phase. 147 students participated in two studies. In the unvaried stimulus study, 71 participants learned the same basic set of 15 exemplars, once(15 × 1 exposure), twice (15 × 2 exposures = 30 total strings) and 3 times (15 × 3 exposures = 45 total strings). In the varied stimulus study, 75 participants learned 15, 30 and 45, all of which were unique, unrepeated exemplars. All groups were asked to classify test strings for their grammaticality following training. Results of the d' measures in the unvaried stimulus study indicate similar performance across the groups. Conversely, the results of the varied stimulus study show that the group presented with 45 unique strings performed significantly better than the baseline group (15 strings). Analysis of the differences across the equivalent groups in the two studies (15 × 2 exposures vs. 30 unique strings and 15 × 3 exposures vs. 45 unique strings) indicates differences in performance only between the group who was presented with the same 15 strings three times and the group presented with 45 unrepeated strings. Taken together, our results shed additional light on the central role of stimulus variation in Artificial Grammar Learning.

Comparing feedforward and recurrent neural network architectures with human behavior in artificial grammar learning

In recent years artificial neural networks achieved performance close to or better than humans in several domains: tasks that were previously human prerogatives, such as language processing, have witnessed remarkable improvements in state of the art models. One advantage of this technological boost is to facilitate comparison between different neural networks and human performance, in order to deepen our understanding of human cognition. Here, we investigate which neural network architecture (feedforward vs. recurrent) matches human behavior in artificial grammar learning, a crucial aspect of language acquisition. Prior experimental studies proved that artificial grammars can be learnt by human subjects after little exposure and often without explicit knowledge of the underlying rules. We tested four grammars with different complexity levels both in humans and in feedforward and recurrent networks. Our results show that both architectures can "learn" (via error back-propagation) the grammars after the same number of training sequences as humans do, but recurrent networks perform closer to humans than feedforward ones, irrespective of the grammar complexity level. Moreover, similar to visual processing, in which feedforward and recurrent architectures have been related to unconscious and conscious processes, the difference in performance between architectures over ten regular grammars shows that simpler and more explicit grammars are better learnt by recurrent architectures, supporting the hypothesis that explicit learning is best modeled by recurrent networks, whereas feedforward networks supposedly capture the dynamics involved in implicit learning.

How does the brain learn environmental structure? Ten core principles for understanding the neurocognitive mechanisms of statistical learning

Despite a growing body of research devoted to the study of how humans encode environmental patterns, there is still no clear consensus about the nature of the neurocognitive mechanisms underpinning statistical learning nor what factors constrain or promote its emergence across individuals, species, and learning situations. Based on a review of research examining the roles of input modality and domain, input structure and complexity, attention, neuroanatomical bases, ontogeny, and phylogeny, ten core principles are proposed. Specifically, there exist two sets of neurocognitive mechanisms underlying statistical learning. First, a "suite" of associative-based, automatic, modality-specific learning mechanisms are mediated by the general principle of cortical plasticity, which results in improved processing and perceptual facilitation of encountered stimuli. Second, an attention-dependent system, mediated by the prefrontal cortex and related attentional and working memory networks, can modulate or gate learning and is necessary in order to learn nonadjacent dependencies and to integrate global patterns across time. This theoretical framework helps clarify conflicting research findings and provides the basis for future empirical and theoretical endeavors.

A New Method for Studying Statistical Learning in Young Children

Purpose The purpose of this study was to determine whether a new oculomotor serial reaction time (RT) task revealed statistical sequence learning in young children. Method We used eye tracking to measure typically developing children's oculomotor RTs in response to cartoon-like creatures that appeared successively in quadrants of a monitor during 200 trials: an initial patterned phase (120 trials) in which the creature's location reflected 15 repetitions of an 8-element sequence, a pseudorandom phase (40 trials) in which the location was not predictable, and a final patterned phase (40 trials). In an auditory-visual version of the task, spoken nonwords linked to quadrants preceded the creature's appearance. In Study 1, we administered either the visual or the auditory-visual version to 5- and 6-year-old children; in Study 2, we examined the performance of 4-year-olds on the auditory-visual version. Results In both studies, group mean RT z scores were significantly shorter ( p < .05) during patterned than pseudorandom phases, with large effect sizes (Cohen's dz values = 1.17-1.79). Conclusion The new oculomotor serial RT task detected statistical sequence learning in typically developing children.

The Promise-and Challenge-of Statistical Learning for Elucidating Atypical Language Development

Statistical learning plays an important role in the acquisition of spoken and written language. It has been proposed that impaired or atypical statistical learning may be linked with language difficulties in developmental disabilities. However, research on statistical learning in individuals with developmental disabilities such as autism spectrum disorder, dyslexia, and specific language impairment, and in individuals with cochlear implants, has produced divergent findings. It is unclear whether, and to what extent, statistical learning is impaired or atypical in each of these developmental disabilities. We suggest that these disparate findings point to several critical issues that must be addressed before we can evaluate the role of statistical learning in atypical child development. While the issues we outline are interrelated, we propose four key points relating to (a) the nature of statistical learning, (b) the myriad of ways in which statistical learning can be measured, (c) our lack of understanding regarding the developmental trajectory of statistical learning, and (d) the role of individual differences. We close by making suggestions that we believe will be helpful in moving the field forward and creating new synergies among researchers, clinicians, and educators to better support language learners.

Visual artificial grammar learning in dyslexia: A meta-analysis

BACKGROUND

Literacy impairments in dyslexia have been hypothesized to be (partly) due to an implicit learning deficit. However, studies of implicit visual artificial grammar learning (AGL) have often yielded null results.

AIMS

The aim of this study is to weigh the evidence collected thus far by performing a meta-analysis of studies on implicit visual AGL in dyslexia.

METHODS AND PROCEDURES

Thirteen studies were selected through a systematic literature search, representing data from 255 participants with dyslexia and 292 control participants (mean age range: 8.5-36.8 years old).

RESULTS

If the 13 selected studies constitute a random sample, individuals with dyslexia perform worse on average than non-dyslexic individuals (average weighted effect size=0.46, 95% CI [0.14 … 0.77], p=0.008), with a larger effect in children than in adults (p=0.041; average weighted effect sizes 0.71 [sig.] versus 0.16 [non-sig.]). However, the presence of a publication bias indicates the existence of missing studies that may well null the effect.

CONCLUSIONS AND IMPLICATIONS

While the studies under investigation demonstrate that implicit visual AGL is impaired in dyslexia (more so in children than in adults, if in adults at all), the detected publication bias suggests that the effect might in fact be zero.

The role of feedback in implicit and explicit artificial grammar learning: a comparison between dyslexic and non-dyslexic adults

The importance of feedback for learning has been firmly established over the past few decades. The question of whether feedback plays a significant role in the statistical learning abilities of adults with dyslexia, however, is currently unresolved. Here, we examined the role of feedback in grammaticality judgment, type of structural knowledge, and confidence rating in both typically developed and dyslexic adults. We implemented two artificial grammar learning experiments: implicit and explicit. The second experiment was directly analogous to the first experiment in all respects except training format: the standard memorization instruction was replaced with an explicit rule-search instruction. Each experiment was conducted with and without performance feedback. While both groups showed significantly improved learning in the feedback-based explicit artificial grammar learning task, only the typically developed adults demonstrated higher levels of conscious structural knowledge. The present study demonstrates that the basis for the grammaticality judgment of adults with dyslexia differs from that of typically developed adults, regardless of increase in the level of explicitness.

Effect of chunk strength on the performance of children with developmental dyslexia on artificial grammar learning task may be related to complexity

There's a long held view that chunks play a crucial role in artificial grammar learning performance. We compared chunk strength influences on performance, in high and low topological entropy (a measure of complexity) grammar systems, with dyslexic children, age-matched and reading-level-matched control participants. Findings show that age-matched control participants' performance reflected equivalent influence of chunk strength in the two topological entropy conditions, as typically found in artificial grammar learning experiments. By contrast, dyslexic children and reading-level-matched controls' performance reflected knowledge of chunk strength only under the low topological entropy condition. In the low topological entropy grammar system, they appeared completely unable to utilize chunk strength to make appropriate test item selections. In line with previous research, this study suggests that for typically developing children, it is the chunks that are attended during artificial grammar learning and create a foundation on which implicit associative learning mechanisms operate, and these chunks are unitized to different strengths. However, for children with dyslexia, it is complexity that may influence the subsequent memorability of chunks, independently of their strength.

Measuring Individual Differences in Implicit Learning with Artificial Grammar Learning Tasks

Abstract. Implicit learning can be defined as learning without intention or awareness. We discuss conceptually and investigate empirically how individual differences in implicit learning can be measured with artificial grammar learning (AGL) tasks. We address whether participants should be instructed to rate the grammaticality or the novelty of letter strings and look at the impact of a knowledge test on measurement quality. We discuss these issues from a conceptual perspective and report three experiments which suggest that (1) the reliability of AGL is moderate and too low for individual assessments, (2) a knowledge test decreases task consistency and increases the correlation with reportable grammar knowledge, and (3) performance in AGL tasks is independent from general intelligence and educational attainment.

Performance of children with developmental dyslexia on high and low topological entropy artificial grammar learning task

Graph complexity as measured by topological entropy has been previously shown to affect performance on artificial grammar learning tasks among typically developing children. The aim of this study was to examine the effect of graph complexity on implicit sequential learning among children with developmental dyslexia. Our goal was to determine whether children's performance depends on the complexity level of the grammar system learned. We conducted two artificial grammar learning experiments that compared performance of children with developmental dyslexia with that of age- and reading level-matched controls. Experiment 1 was a high topological entropy artificial grammar learning task that aimed to establish implicit learning phenomena in children with developmental dyslexia using previously published experimental conditions. Experiment 2 is a lower topological entropy variant of that task. Results indicated that given a high topological entropy grammar system, children with developmental dyslexia who were similar to the reading age-matched control group had substantial difficulty in performing the task as compared to typically developing children, who exhibited intact implicit learning of the grammar. On the other hand, when tested on a lower topological entropy grammar system, all groups performed above chance level, indicating that children with developmental dyslexia were able to identify rules from a given grammar system. The results reinforced the significance of graph complexity when experimenting with artificial grammar learning tasks, particularly with dyslexic participants.

Implicit learning is order dependent

We report two experiments using the artificial-grammar task that demonstrate order dependence in implicit learning. Studying grammatical training strings in different orders did not affect participants' discrimination of grammatical from ungrammatical test strings, but it did affect their judgments about specific test strings. Current accounts of learning in the artificial-grammar task focus on category-level discrimination and largely ignore item-level discrimination. Hence, the results highlight the importance of moving theory from a category- to an item-level of analysis and point to a new way to evaluate and to refine accounts of implicit learning.