Structured Sequence Learning: Animal Abilities, Cognitive Operations, and Language Evolution

Abstract processing of syllabic structures in early infancy

Syllables are one of the fundamental building blocks of early language acquisition. From birth onwards, infants preferentially segment, process and represent the speech into syllable-sized units, raising the question of what type of computations infants are able to perform on these perceptual units. Syllables are abstract units structured in a way that allows grouping phonemes into sequences. The goal of this research was to investigate 4-to-5-month-old infants' ability to encode the internal structure of syllables, at a target age when the language system is not yet specialized on the sounds and the phonotactics of native languages. We conducted two experiments in which infants were first familiarized to lists of syllables implementing either CVC (consonant-vowel-consonant) or CCV (consonant-consonant-vowel) structures, then presented with new syllables implementing both structures at test. Experiments differ in the degree of phonological similarity between the materials used at familiarization and test. Results show that infants were able to differentiate syllabic structures at test, even when test syllables were implemented by combinations of phonemes that infants did not hear before. Only infants familiarized with CVC syllables discriminated the structures at test, pointing to a processing advantage for CVC over CCV structures. This research shows that, in addition to preferentially processing the speech into syllable-sized units, during the first months of life, infants are also capable of performing fine-grained computations within such units.

Implicit auditory memory in older listeners: From encoding to 6-month retention

Any listening task, from sound recognition to sound-based communication, rests on auditory memory which is known to decline in healthy ageing. However, how this decline maps onto multiple components and stages of auditory memory remains poorly characterised. In an online unsupervised longitudinal study, we tested ageing effects on implicit auditory memory for rapid tone patterns. The test required participants (younger, aged 20-30, and older adults aged 60-70) to quickly respond to rapid regularly repeating patterns emerging from random sequences. Patterns were novel in most trials (REGn), but unbeknownst to the participants, a few distinct patterns reoccurred identically throughout the sessions (REGr). After correcting for processing speed, the response times (RT) to REGn should reflect the information held in echoic and short-term memory before detecting the pattern; long-term memory formation and retention should be reflected by the RT advantage (RTA) to REGr vs REGn which is expected to grow with exposure. Older participants were slower than younger adults in detecting REGn and exhibited a smaller RTA to REGr. Computational simulations using a model of auditory sequence memory indicated that these effects reflect age-related limitations both in early and long-term memory stages. In contrast to ageing-related accelerated forgetting of verbal material, here older adults maintained stable memory traces for REGr patterns up to 6 months after the first exposure. The results demonstrate that ageing is associated with reduced short-term memory and long-term memory formation for tone patterns, but not with forgetting, even over surprisingly long timescales.

Do they know or just do it? Investigating implicit and explicit sequence learning by capuchin monkeys, human adults and children

In humans, it is now established that sequential regularities can be learned implicitly (i.e. without acquiring conscious knowledge) or explicitly (with acquisition of conscious knowledge). Is this dual-processing capability also the case for non-human primates? In this study, we designed a non-verbal task to probe implicit and explicit sequence learning in capuchin monkeys (Sapajus sp., n = 12), human adults (n = 12), and children from 5 to 10 years old (n = 64). After learning spatial sequences on a touchscreen, participants' conscious access to the sequences was probed with a forced choice sequence completion test. All performed above chance level in this test, without being instructed or trained to do so. However, only human adults who reported the presence of regularities performed at ceiling level. We suggest future directions that could build on our findings to disentangle implicit and explicit learning in monkeys and children.

Exploring the neurobiology of Merge at a basic level: insights from a novel artificial grammar paradigm

Introduction

Human language allows us to generate an infinite number of linguistic expressions. It's proposed that this competence is based on a binary syntactic operation, Merge, combining two elements to form a new constituent. An increasing number of recent studies have shifted from complex syntactic structures to two-word constructions to investigate the neural representation of this operation at the most basic level.

Methods

This fMRI study aimed to develop a highly flexible artificial grammar paradigm for testing the neurobiology of human syntax at a basic level. During scanning, participants had to apply abstract syntactic rules to assess whether a given two-word artificial phrase could be further merged with a third word. To control for lower-level template-matching and working memory strategies, an additional non-mergeable word-list task was set up.

Results

Behavioral data indicated that participants complied with the experiment. Whole brain and region of interest (ROI) analyses were performed under the contrast of "structure > word-list." Whole brain analysis confirmed significant involvement of the posterior inferior frontal gyrus [pIFG, corresponding to Brodmann area (BA) 44]. Furthermore, both the signal intensity in Broca's area and the behavioral performance showed significant correlations with natural language performance in the same participants. ROI analysis within the language atlas and anatomically defined Broca's area revealed that only the pIFG was reliably activated.

Discussion

Taken together, these results support the notion that Broca's area, particularly BA 44, works as a combinatorial engine where words are merged together according to syntactic information. Furthermore, this study suggests that the present artificial grammar may serve as promising material for investigating the neurobiological basis of syntax, fostering future cross-species studies.

Coevolution of language and tools in the human brain: An ALE meta-analysis of neural activation during syntactic processing and tool use

Language and complex tool use are often cited as behaviors unique to humans and may be evolutionarily linked owing to the underlying cognitive processes they have in common. We executed a quantitative activation likelihood estimation (ALE) meta-analysis (GingerALE 2.3) on published, whole-brain neuroimaging studies to identify areas associated with syntactic processing and/or tool use in humans. Significant clusters related to syntactic processing were identified in areas known to be related to language production and comprehension, including bilateral Broca's area in the inferior frontal gyrus. Tool use activation clusters were all in the left hemisphere and included the primary motor cortex and premotor cortex, in addition to other areas involved with sensorimotor transformation. Activation shared by syntactic processing and tool use was only significant at one cluster, located in the pars opercularis of the left inferior frontal gyrus. This minimal overlap between syntactic processing and tool use activation from our meta-analysis of neuroimaging studies indicates that there is not a widespread common neural network between the two. Broca's area may serve as an important hub that was initially recruited in early human evolution in the context of simple tool use, but was eventually co-opted for linguistic purposes, including the sequential and hierarchical ordering processes that characterize syntax. In the future, meta-analyses of additional components of language may allow for a more comprehensive examination of the functional networks that underlie the coevolution of human language and complex tool use.

Mandarin Chinese L1 and L2 complex sentence reading reveals a consistent electrophysiological pattern of highly interactive syntactic and semantic processing: An ERP study

Introduction

A hallmark of the human language faculty is processing complex hierarchical syntactic structures across languages. However, for Mandarin Chinese, a language typically dependent on semantic combinations and free of morphosyntactic information, the relationship between syntactic and semantic processing during Chinese complex sentence reading is unclear. From the neuropsychological perspective of bilingual studies, whether second language (L2) learners can develop a consistent pattern of target language (i.e., L2) comprehension regarding the interplay of syntactic and semantic processing, especially when their first language (L1) and L2 are typologically distinct, remains to be determined. In this study, Chinese complex sentences with center-embedded relative clauses were generated. By utilizing the high-time-resolution technique of event-related potentials (ERPs), this study aimed to investigate the processing relationships between syntactic and semantic information during Chinese complex sentence reading in both Chinese L1 speakers and highly proficient L2 learners from South Korea.

Methods

Normal, semantically violated (SEM), and double-violated (containing both semantic and syntactic violations, SEM + SYN) conditions were set with regard to the nonadjacent dependencies of the Chinese complex sentence, and participants were required to judge whether the sentences they read were acceptable.

Results

The ERP results showed that sentences with "SEM + SYN" did not elicit early left anterior negativity (ELAN), a component assumed to signal initial syntactic processing, but evoked larger components in the N400 and P600 windows than those of the "SEM" condition, thus exhibiting a biphasic waveform pattern consistent for both groups and in line with previous studies using simpler Chinese syntactic structures. The only difference between the L1 and L2 groups was that L2 learners presented later latencies of the corresponding ERP components.

Discussion

Taken together, these results do not support the temporal and functional priorities of syntactic processing as identified in morphologically rich languages (e.g., German) and converge on the notion that even for Chinese complex sentence reading, syntactic and semantic processing are highly interactive. This is consistent across L1 speakers and high-proficiency L2 learners with typologically different language backgrounds.

Strengths and challenges of longitudinal non-human primate neuroimaging

Longitudinal non-human primate neuroimaging has the potential to greatly enhance our understanding of primate brain structure and function. Here we describe its specific strengths, compared to both cross-sectional non-human primate neuroimaging and longitudinal human neuroimaging, but also its associated challenges. We elaborate on factors guiding the use of different analytical tools, subject-specific versus age-specific templates for analyses, and issues related to statistical power.

The evolution of hierarchical structure building capacity for language and music: a bottom-up perspective

A central property of human language is its hierarchical structure. Humans can flexibly combine elements to build a hierarchical structure expressing rich semantics. A hierarchical structure is also considered as playing a key role in many other human cognitive domains. In music, auditory-motor events are combined into hierarchical pitch and/or rhythm structure expressing affect. How did such a hierarchical structure building capacity evolve? This paper investigates this question from a bottom-up perspective based on a set of action-related components as a shared basis underlying cognitive capacities of nonhuman primates and humans. Especially, I argue that the evolution of hierarchical structure building capacity for language and music is tractable for comparative evolutionary study once we focus on the gradual elaboration of shared brain architecture: the cortico-basal ganglia-thalamocortical circuits for hierarchical control of goal-directed action and the dorsal pathways for hierarchical internal models. I suggest that this gradual elaboration of the action-related brain architecture in the context of vocal control and tool-making went hand in hand with amplification of working memory, and made the brain ready for hierarchical structure building in language and music.

Hierarchical syntactic processing is beyond mere associating: Functional magnetic resonance imaging evidence from a novel artificial grammar

Grammar is central to any natural language. In the past decades, the artificial grammar of the AⁿBⁿ type in which a pair of associated elements can be nested in the other pair was considered as a desirable model to mimic human language syntax without semantic interference. However, such a grammar relies on mere associating mechanisms, thus insufficient to reflect the hierarchical nature of human syntax. Here, we test how the brain imposes syntactic hierarchies according to the category relations on linearized sequences by designing a novel artificial “Hierarchical syntactic structure‐building Grammar” (HG), and compare this to the AⁿBⁿ grammar as a “Nested associating Grammar” (NG) based on multilevel associations. Thirty‐six healthy German native speakers were randomly assigned to one of the two grammars. Both groups performed a grammaticality judgment task on auditorily presented word sequences generated by the corresponding grammar in the scanner after a successful explicit behavioral learning session. Compared to the NG group, we found that the HG group showed a (a) significantly higher involvement of Brodmann area (BA) 44 in Broca's area and the posterior superior temporal gyrus (pSTG); and (b) qualitatively distinct connectivity between the two regions. Thus, the present study demonstrates that the build‐up process of syntactic hierarchies on the basis of category relations critically relies on a distinctive left‐hemispheric syntactic network involving BA 44 and pSTG. This indicates that our novel artificial grammar can constitute a suitable experimental tool to investigate syntax‐specific processes in the human brain.

Language and action in Broca's area: Computational differentiation and cortical segregation

Actions have been proposed to follow hierarchical principles similar to those hypothesized for language syntax. These structural similarities are claimed to be reflected in the common involvement of certain neural populations of Broca's area, in the Inferior Frontal Gyrus (IFG). In this position paper, we follow an influential hypothesis in linguistic theory to introduce the syntactic operation Merge and the corresponding motor/conceptual interfaces. We argue that actions hierarchies do not follow the same principles ruling language syntax. We propose that hierarchy in the action domain lies in predictive processing mechanisms mapping sensory inputs and statistical regularities of action-goal relationships. At the cortical level, distinct Broca's subregions appear to support different types of computations across the two domains. We argue that anterior BA44 is a major hub for the implementation of the syntactic operation Merge. On the other hand, posterior BA44 is recruited in selecting premotor mental representations based on the information provided by contextual signals. This functional distinction is corroborated by a recent meta-analysis (Papitto, Friederici, & Zaccarella, 2020). We conclude by suggesting that action and language can meet only where the interfaces transfer abstract computations either to the external world or to the internal mental world.

Editors' Review and Introduction: Learning Grammatical Structures: Developmental, Cross-Species, and Computational Approaches

Human languages all have a grammar, that is, rules that determine how symbols in a language can be combined to create complex meaningful expressions. Despite decades of research, the evolutionary, developmental, cognitive, and computational bases of grammatical abilities are still not fully understood. "Artificial Grammar Learning" (AGL) studies provide important insights into how rules and structured sequences are learned, the relevance of these processes to language in humans, and whether the cognitive systems involved are shared with other animals. AGL tasks can be used to study how human adults, infants, animals, or machines learn artificial grammars of various sorts, consisting of rules defined typically over syllables, sounds, or visual items. In this introduction, we distill some lessons from the nine other papers in this special issue, which review the advances made from this growing body of literature. We provide a critical synthesis, identify the questions that remain open, and recognize the challenges that lie ahead. A key observation across the disciplines is that the limits of human, animal, and machine capabilities have yet to be found. Thus, this interdisciplinary area of research firmly rooted in the cognitive sciences has unearthed exciting new questions and venues for research, along the way fostering impactful collaborations between traditionally disconnected disciplines that are breaking scientific ground.

Structured Sequence Learning: Animal Abilities, Cognitive Operations, and Language Evolution

Human language is a salient example of a neurocognitive system that is specialized to process complex dependencies between sensory events distributed in time, yet how this system evolved and specialized remains unclear. Artificial Grammar Learning (AGL) studies have generated a wealth of insights into how human adults and infants process different types of sequencing dependencies of varying complexity. The AGL paradigm has also been adopted to examine the sequence processing abilities of nonhuman animals. We critically evaluate this growing literature in species ranging from mammals (primates and rats) to birds (pigeons, songbirds, and parrots) considering also cross-species comparisons. The findings are contrasted with seminal studies in human infants that motivated the work in nonhuman animals. This synopsis identifies advances in knowledge and where uncertainty remains regarding the various strategies that nonhuman animals can adopt for processing sequencing dependencies. The paucity of evidence in the few species studied to date and the need for follow-up experiments indicate that we do not yet understand the limits of animal sequence processing capacities and thereby the evolutionary pattern. This vibrant, yet still budding, field of research carries substantial promise for advancing knowledge on animal abilities, cognitive substrates, and language evolution.