Evolutionary origins of non-adjacent sequence processing in primate brain potentials

Neural evidence for referential understanding of object words in dogs

Using words to refer to objects in the environment is a core feature of the human language faculty. Referential understanding assumes the formation of mental representations of these words.1,2 Such understanding of object words has not yet been demonstrated as a general capacity in any non-human species,3 despite multiple behavior-based case reports.4,5,6,7,8,9,10 In human event-related potential (ERP) studies, object word knowledge is typically tested using the semantic violation paradigm, where words are presented either with their referent (match) or another object (mismatch).11,12 Such mismatch elicits an N400 effect, a well-established neural correlate of semantic processing.12,13 Reports of preverbal infant N400 evoked by semantic violations14 assert the use of this paradigm to probe mental representations of object words in nonverbal populations. Here, measuring dogs' (Canis familiaris) ERPs to objects primed with matching or mismatching object words, we found a mismatch effect at a frontal electrode, with a latency (206-606 ms) comparable to the human N400. A greater difference for words that dogs knew better, according to owner reports, further supported a semantic interpretation of this effect. Semantic expectations emerged irrespective of vocabulary size, demonstrating the prevalence of referential understanding in dogs. These results provide the first neural evidence for object word knowledge in a non-human animal. VIDEO ABSTRACT.

A Cortical Mechanism Linking Saliency Detection and Motor Reactivity in Rhesus Monkeys

Sudden and surprising sensory events trigger neural processes that swiftly adjust behavior. To study the phylogenesis and the mechanism of this phenomenon, we trained two male rhesus monkeys to keep a cursor inside a visual target by exerting force on an isometric joystick. We examined the effect of surprising auditory stimuli on exerted force, scalp electroencephalographic (EEG) activity, and local field potentials (LFPs) recorded from the dorsolateral prefrontal cortex. Auditory stimuli elicited (1) a biphasic modulation of isometric force, a transient decrease followed by a corrective tonic increase, and (2) EEG and LFP deflections dominated by two large negative-positive waves (N70 and P130). The EEG potential was symmetrical and maximal at the scalp vertex, highly reminiscent of the human "vertex potential." Electrocortical potentials and force were tightly coupled: the P130 amplitude predicted the magnitude of the corrective force increase, particularly in the LFPs recorded from deep rather than superficial cortical layers. These results disclose a phylogenetically preserved corticomotor mechanism supporting adaptive behavior in response to salient sensory events.Significance Statement Survival in the natural world depends on an animal's capacity to adapt ongoing behavior to abrupt unexpected events. To study the neural mechanisms underlying this capacity, we trained monkeys to apply constant force on a joystick while we recorded their brain activity from the scalp and the prefrontal cortex contralateral to the hand holding the joystick. Unexpected auditory stimuli elicited a biphasic force modulation: a transient reduction followed by a corrective adjustment. The same stimuli also elicited EEG and LFP responses, dominated by a biphasic wave that predicted the magnitude of the behavioral adjustment. These results disclose a phylogenetically preserved corticomotor mechanism supporting adaptive behavior in response to unexpected events.

A Special Role of Syllables, But Not Vowels or Consonants, for Nonadjacent Dependency Learning

Successful language processing entails tracking (morpho)syntactic relationships between distant units of speech, so-called nonadjacent dependencies (NADs). Many cues to such dependency relations have been identified, yet the linguistic elements encoding them have received little attention. In the present investigation, we tested whether and how these elements, here syllables, consonants, and vowels, affect behavioral learning success as well as learning-related changes in neural activity in relation to item-specific NAD learning. In a set of two EEG studies with adults, we compared learning under conditions where either all segment types (Experiment 1) or only one segment type (Experiment 2) was informative. The collected behavioral and ERP data indicate that, when all three segment types are available, participants mainly rely on the syllable for NAD learning. With only one segment type available for learning, adults also perform most successfully with syllable-based dependencies. Although we find no evidence for successful learning across vowels in Experiment 2, dependencies between consonants seem to be identified at least passively at the phonetic-feature level. Together, these results suggest that successful item-specific NAD learning may depend on the availability of syllabic information. Furthermore, they highlight consonants' distinctive power to support lexical processes. Although syllables show a clear facilitatory function for NAD learning, the underlying mechanisms of this advantage require further research.

Chimpanzees produce diverse vocal sequences with ordered and recombinatorial properties

The origins of human language remains a major question in evolutionary science. Unique to human language is the capacity to flexibly recombine a limited sound set into words and hierarchical sequences, generating endlessly new sentences. In contrast, sequence production of other animals appears limited, stunting meaning generation potential. However, studies have rarely quantified flexibility and structure of vocal sequence production across the whole repertoire. Here, we used such an approach to examine the structure of vocal sequences in chimpanzees, known to combine calls used singly into longer sequences. Focusing on the structure of vocal sequences, we analysed 4826 recordings of 46 wild adult chimpanzees from Taï National Park. Chimpanzees produced 390 unique vocal sequences. Most vocal units emitted singly were also emitted in two-unit sequences (bigrams), which in turn were embedded into three-unit sequences (trigrams). Bigrams showed positional and transitional regularities within trigrams with certain bigrams predictably occurring in either head or tail positions in trigrams, and predictably co-occurring with specific other units. From a purely structural perspective, the capacity to organize single units into structured sequences offers a versatile system potentially suitable for expansive meaning generation. Further research must show to what extent these structural sequences signal predictable meanings.

Bridging the Gap Between Neurons and Cognition Through Assemblies of Neurons

During recent decades, our understanding of the brain has advanced dramatically at both the cellular and molecular levels and at the cognitive neurofunctional level; however, a huge gap remains between the microlevel of physiology and the macrolevel of cognition. We propose that computational models based on assemblies of neurons can serve as a blueprint for bridging these two scales. We discuss recently developed computational models of assemblies that have been demonstrated to mediate higher cognitive functions such as the processing of simple sentences, to be realistically realizable by neural activity, and to possess general computational power.

Arc-shaped pitch contours facilitate item recognition in non-human animals

Acoustic changes linked to natural prosody are a key source of information about the organization of language. Both human infants and adults readily take advantage of such changes to discover and memorize linguistic patterns. Do they so because our brain is efficiently wired to specifically process linguistic stimuli? Or are we co-opting for language acquisition purposes more general principles that might be inherited from our animal ancestors? Here, we address this question by exploring if other species profit from prosody to better process acoustic sequences. More specifically, we test whether arc-shaped pitch contours defining natural prosody might facilitate item recognition and memorization in rats. In two experiments, we presented to the rats nonsense words with flat, natural, inverted and random prosodic contours. We observed that the animals correctly recognized the familiarization words only when arc-shaped pitch contours were implemented over them. Our results suggest that other species might also benefit from prosody for the memorization of items in a sequence. Such capacity seems to be rooted in general principles of how biological sounds are produced and processed.

Mini Pinyin: A modified miniature language for studying language learning and incremental sentence processing

Artificial grammar learning (AGL) paradigms are used extensively to characterise (neuro)cognitive bases of language learning. However, despite their effectiveness in characterising the capacity to learn complex structured sequences, AGL paradigms lack ecological validity and typically do not account for cross-linguistic differences in sentence comprehension. Here, we describe a new modified miniature language paradigm - Mini Pinyin - that mimics natural language as it is based on an existing language (Mandarin Chinese) and includes both structure and meaning. Mini Pinyin contains a number of cross-linguistic elements, including varying word orders and classifier-noun rules. To evaluate the effectiveness of Mini Pinyin, 76 (mean age = 24.9; 26 female) monolingual native English speakers completed a learning phase followed by a sentence acceptability judgement task. Generalised mixed effects modelling revealed that participants attained a moderate degree of accuracy on the judgement task, with performance scores ranging from 25% to 100% accuracy depending on the word order of the sentence. Further, sentences compatible with the canonical English word order were learned more efficiently than non-canonical word orders. We controlled for inter-individual differences in statistical learning ability, which accounted for ~20% of the variance in performance on the sentence judgement task. We provide stimuli and statistical analysis scripts as open-source resources and discuss how future research can utilise this paradigm to study the neurobiological basis of language learning. Mini Pinyin affords a convenient tool for improving the future of language learning research by building on the parameters of traditional AGL or existing miniature language paradigms.

Linguistic and non-linguistic non-adjacent dependency learning in early development

Non-adjacent dependencies (NADs) are important building blocks for language and extracting them from the input is a fundamental part of language acquisition. Prior event-related potential (ERP) studies revealed changes in the neural signature of NAD learning between infancy and adulthood, suggesting a developmental shift in the learning route for NADs. The present study aimed to specify which brain regions are involved in this developmental shift and whether this shift extends to NAD learning in the non-linguistic domain. In two experiments, 2- and 3-year-old German-learning children were familiarized with either Italian sentences or tone sequences containing NADs and subsequently tested with NAD violations, while functional near-infrared spectroscopy (fNIRS) data were recorded. Results showed increased hemodynamic responses related to the detection of linguistic NAD violations in the left temporal, inferior frontal, and parietal regions in 2-year-old children, but not in 3-year-old children. A different developmental trajectory was found for non-linguistic NADs, where 3-year-old, but not 2-year-old children showed evidence for the detection of non-linguistic NAD violations. These results confirm a developmental shift in the NAD learning route and point to distinct mechanisms underlying NAD learning in the linguistic and the non-linguistic domain.

Nonadjacent dependency processing in monkeys, apes, and humans

The ability to track syntactic relationships between words, particularly over distances ("nonadjacent dependencies"), is a critical faculty underpinning human language, although its evolutionary origins remain poorly understood. While some monkey species are reported to process auditory nonadjacent dependencies, comparative data from apes are missing, complicating inferences regarding shared ancestry. Here, we examined nonadjacent dependency processing in common marmosets, chimpanzees, and humans using "artificial grammars": strings of arbitrary acoustic stimuli composed of adjacent (nonhumans) or nonadjacent (all species) dependencies. Individuals from each species (i) generalized the grammars to novel stimuli and (ii) detected grammatical violations, indicating that they processed the dependencies between constituent elements. Furthermore, there was no difference between marmosets and chimpanzees in their sensitivity to nonadjacent dependencies. These notable similarities between monkeys, apes, and humans indicate that nonadjacent dependency processing, a crucial cognitive facilitator of language, is an ancestral trait that evolved at least ~40 million years before language itself.

Non-adjacent Dependency Learning in Humans and Other Animals

Learning and processing natural language requires the ability to track syntactic relationships between words and phrases in a sentence, which are often separated by intervening material. These nonadjacent dependencies can be studied using artificial grammar learning paradigms and structured sequence processing tasks. These approaches have been used to demonstrate that human adults, infants and some nonhuman animals are able to detect and learn dependencies between nonadjacent elements within a sequence. However, learning nonadjacent dependencies appears to be more cognitively demanding than detecting dependencies between adjacent elements, and only occurs in certain circumstances. In this review, we discuss different types of nonadjacent dependencies in language and in artificial grammar learning experiments, and how these differences might impact learning. We summarize different types of perceptual cues that facilitate learning, by highlighting the relationship between dependent elements bringing them closer together either physically, attentionally, or perceptually. Finally, we review artificial grammar learning experiments in human adults, infants, and nonhuman animals, and discuss how similarities and differences observed across these groups can provide insights into how language is learned across development and how these language-related abilities might have evolved.

Common marmosets (Callithrix jacchus) cannot recognize global configurations of sound patterns but can recognize adjacent relations of sounds

Processing the temporal configuration of discrete sounds to extract a regular pattern is fundamental to humans' faculties of perceiving words and musical phrases. To investigate such auditory pattern perception in monkeys, I trained two common marmosets to discriminate between AB-AB and AA-BB patterns under two training paradigms. One was an absolute discrimination task, in which the discrimination between these stimuli without reference cues was required. The other was a relative discrimination task, in which the detection of a change from one stimulus to the other was required. The marmosets failed in the absolute discrimination task but achieved the relative discrimination task. Failure in the absolute task indicated that the marmosets were unable to form a representation of the global sound patterns in their long-term memory stores. In contrast, success in the relative task indicated that the marmosets had short-term memory of ongoing sounds that enabled an online monitoring to detect deviations between incoming sounds and the anticipated upcoming sounds. Thus, the current findings imply that marmosets can at least perceive adjacent tone relations in an auditory stream regardless of the temporal configuration of the global sound patterns.

How does the brain learn environmental structure? Ten core principles for understanding the neurocognitive mechanisms of statistical learning

Despite a growing body of research devoted to the study of how humans encode environmental patterns, there is still no clear consensus about the nature of the neurocognitive mechanisms underpinning statistical learning nor what factors constrain or promote its emergence across individuals, species, and learning situations. Based on a review of research examining the roles of input modality and domain, input structure and complexity, attention, neuroanatomical bases, ontogeny, and phylogeny, ten core principles are proposed. Specifically, there exist two sets of neurocognitive mechanisms underlying statistical learning. First, a "suite" of associative-based, automatic, modality-specific learning mechanisms are mediated by the general principle of cortical plasticity, which results in improved processing and perceptual facilitation of encountered stimuli. Second, an attention-dependent system, mediated by the prefrontal cortex and related attentional and working memory networks, can modulate or gate learning and is necessary in order to learn nonadjacent dependencies and to integrate global patterns across time. This theoretical framework helps clarify conflicting research findings and provides the basis for future empirical and theoretical endeavors.

Hierarchy processing in human neurobiology: how specific is it?

Although human and non-human animals share a number of perceptual and cognitive abilities, they differ in their ability to process hierarchically structured sequences. This becomes most evident in the human capacity to process natural language characterized by structural hierarchies. This capacity is neuroanatomically grounded in the posterior part of left Broca's area (Brodmann area (BA) 44), located in the inferior frontal gyrus, and its dorsal white matter fibre connection to the temporal cortex. Within this neural network, BA 44 itself subserves hierarchy building and the strength of its connection to the temporal cortex correlates with the processing of syntactically complex sentences. Whether these brain structures are also relevant for other human cognitive abilities is a current debate. Here, this question will be evaluated with respect to those human cognitive abilities that are assumed to require hierarchy building, such as music, mathematics and Theory of Mind. Rather than supporting a domain-general view, the data indicate domain-selective neural networks as the neurobiological basis for processing hierarchy in different cognitive domains. Recent cross-species white matter comparisons suggest that particular connections within the networks may make the crucial difference in the brain structure of human and non-human primates, thereby enabling cognitive functions specific to humans. This article is part of the theme issue 'What can animal communication teach us about human language?'

Systems Underlying Human and Old World Monkey Communication: One, Two, or Infinite

Using artificially synthesized stimuli, previous research has shown that cotton-top tamarin monkeys easily learn simple AB grammar sequences, but not the more complex AnBn sequences that require hierarchical structure. Humans have no trouble learning AnBn combinations. A more recent study, using similar artificially created stimuli, showed that there is a neuroanatomical difference in the brain between these two kinds of arrays. While the simpler AB sequences recruit the frontal operculum, the AnBn array recruits the phylogenetically newer Broca's area. We propose that on close inspection, reported vocal repertoires of Old World Monkeys show that these nonhuman primates are capable of calls that have two items in them, but never more than two. These are simple AB sequences, as predicted by previous research. In addition, we suggest the two-item call cannot be the result of a combinatorial operation that we see in human language, where the recursive operation of Merge allows for a potentially infinite array of structures. In our view, the two-item calls of nonhuman primates result from a dual-compartment frame into which each of the calls can fit without having to be combined by an operation such as Merge.

Positional encoding in cotton-top tamarins (Saguinus oedipus)

Strategies used in artificial grammar learning can shed light into the abilities of different species to extract regularities from the environment. In the A(X)ⁿB rule, A and B items are linked, but assigned to different positional categories and separated by distractor items. Open questions are how widespread is the ability to extract positional regularities from A(X)ⁿB patterns, which strategies are used to encode positional regularities and whether individuals exhibit preferences for absolute or relative position encoding. We used visual arrays to investigate whether cotton-top tamarins (Saguinusoedipus) can learn this rule and which strategies they use. After training on a subset of exemplars, two of the tested monkeys successfully generalized to novel combinations. These tamarins discriminated between categories of tokens with different properties (A, B, X) and detected a positional relationship between non-adjacent items even in the presence of novel distractors. The pattern of errors revealed that successful subjects used visual similarity with training stimuli to solve the task and that successful tamarins extracted the relative position of As and Bs rather than their absolute position, similarly to what has been observed in other species. Relative position encoding appears to be favoured in different tasks and taxa. Generalization, though, was incomplete, since we observed a failure with items that during training had always been presented in reinforced arrays, showing the limitations in grasping the underlying positional rule. These results suggest the use of local strategies in the extraction of positional rules in cotton-top tamarins.

Capacities and neural mechanisms for auditory statistical learning across species

Statistical learning has been proposed as a possible mechanism by which individuals can become sensitive to the structures of language fundamental for speech perception. Since its description in human infants, statistical learning has been described in human adults and several non-human species as a general process by which animals learn about stimulus-relevant statistics. The neurobiology of statistical learning is beginning to be understood, but many questions remain about the underlying mechanisms. Why is the developing brain particularly sensitive to stimulus and environmental statistics, and what neural processes are engaged in the adult brain to enable learning from statistical regularities in the absence of external reward or instruction? This review will survey the statistical learning abilities of humans and non-human animals with a particular focus on communicative vocalizations. We discuss the neurobiological basis of statistical learning, and specifically what can be learned by exploring this process in both humans and laboratory animals. Finally, we describe advantages of studying vocal communication in rodents as a means to further our understanding of the cortical plasticity mechanisms engaged during statistical learning. We examine the use of rodents in the context of pup retrieval, which is an auditory-based and experience-dependent form of maternal behavior.

Auditory sequence perception in common marmosets (Callithrix jacchus)

One of the essential linguistic and musical faculties of humans is the ability to recognize the structure of sound configurations and to extract words and melodies from continuous sound sequences. However, monkeys' ability to process the temporal structure of sounds is controversial. Here, to investigate whether monkeys can analyze the temporal structure of auditory patterns, two common marmosets were trained to discriminate auditory patterns in three experiments. In Experiment 1, the marmosets were able to discriminate trains of either 0.5- or 2-kHz tones repeated in either 50- or 200-ms intervals. However, the marmosets were not able to discriminate ABAB from AABB patterns consisting of A (0.5-kHz/50-ms pulse) and B (2-kHz/200-ms pulse) elements in Experiment 2, and A (0.5-kHz/50-ms pulse) and B (0.5-kHz/200-ms pulse) [or A (0.5-kHz/200-ms pulse) and B (2-kHz/200-ms pulse)] in Experiment 3. Consequently, the results indicated that the marmosets could not perceive tonal structures in terms of the temporal configuration of discrete sounds, whereas they could recognize the acoustic features of the stimuli. The present findings were supported by cognitive and brain studies that indicated a limited ability to process sound sequences. However, more studies are needed to confirm the ability of auditory sequence perception in common marmosets.

Infant cognition includes the potentially human-unique ability to encode embedding

Human cognition relies on the ability to encode complex regularities in the input. Regularities above a certain complexity level can involve the feature of embedding, defined by nested relations between sequential elements. While comparative studies suggest the cognitive processing of embedding to be human specific, evidence of its ontogenesis is lacking. To assess infants' ability to process embedding, we implemented nested relations in tone sequences, minimizing perceptual and memory requirements. We measured 5-month-olds' brain responses in two auditory oddball paradigms, presenting standard sequences with one or two levels of embedding, interspersed with infrequent deviant sequences violating the established embedding rules. Brain potentials indicate that infants detect embedding violations and thus appear to track nested relations. This shows that the ability to encode embedding may be part of the basic human cognitive makeup, which might serve as scaffolding for the acquisition of complex regularities in language or music.

Auditory and Visual Sequence Learning in Humans and Monkeys using an Artificial Grammar Learning Paradigm

Language flexibly supports the human ability to communicate using different sensory modalities, such as writing and reading in the visual modality and speaking and listening in the auditory domain. Although it has been argued that nonhuman primate communication abilities are inherently multisensory, direct behavioural comparisons between human and nonhuman primates are scant. Artificial grammar learning (AGL) tasks and statistical learning experiments can be used to emulate ordering relationships between words in a sentence. However, previous comparative work using such paradigms has primarily investigated sequence learning within a single sensory modality. We used an AGL paradigm to evaluate how humans and macaque monkeys learn and respond to identically structured sequences of either auditory or visual stimuli. In the auditory and visual experiments, we found that both species were sensitive to the ordering relationships between elements in the sequences. Moreover, the humans and monkeys produced largely similar response patterns to the visual and auditory sequences, indicating that the sequences are processed in comparable ways across the sensory modalities. These results provide evidence that human sequence processing abilities stem from an evolutionarily conserved capacity that appears to operate comparably across the sensory modalities in both human and nonhuman primates. The findings set the stage for future neurobiological studies to investigate the multisensory nature of these sequencing operations in nonhuman primates and how they compare to related processes in humans.

Evolutionarily conserved neural signatures involved in sequencing predictions and their relevance for language

Predicting the occurrence of future events from prior ones is vital for animal perception and cognition. Although how such sequence learning (a form of relational knowledge) relates to particular operations in language remains controversial, recent evidence shows that sequence learning is disrupted in frontal lobe damage associated with aphasia. Also, neural sequencing predictions at different temporal scales resemble those involved in language operations occurring at similar scales. Furthermore, comparative work in humans and monkeys highlights evolutionarily conserved frontal substrates and predictive oscillatory signatures in the temporal lobe processing learned sequences of speech signals. Altogether this evidence supports a relational knowledge hypothesis of language evolution, proposing that language processes in humans are functionally integrated with an ancestral neural system for predictive sequence learning.

Non-adjacent Dependencies Processing in Human and Non-human Primates

Human and non-human primates share the ability to extract adjacent dependencies and, under certain conditions, non-adjacent dependencies (i.e., predictive relationships between elements that are separated by one or several intervening elements in a sequence). In this study, we explore the online extraction dynamics of non-adjacent dependencies in humans and baboons using a serial reaction time task. Participants had to produce three-target sequences containing deterministic relationships between the first and last target locations. In Experiment 1, participants from the two species could extract these non-adjacent dependencies, but humans required less exposure than baboons. In Experiment 2, the data show for the first time in a non-human primate species the successful generalization of sequential non-adjacent dependencies over novel intervening items. These findings provide new evidence to further constrain current theories about the nature and the evolutionary origins of the learning mechanisms allowing the extraction of non-adjacent dependencies.

Sequence learning modulates neural responses and oscillatory coupling in human and monkey auditory cortex

Learning complex ordering relationships between sensory events in a sequence is fundamental for animal perception and human communication. While it is known that rhythmic sensory events can entrain brain oscillations at different frequencies, how learning and prior experience with sequencing relationships affect neocortical oscillations and neuronal responses is poorly understood. We used an implicit sequence learning paradigm (an "artificial grammar") in which humans and monkeys were exposed to sequences of nonsense words with regularities in the ordering relationships between the words. We then recorded neural responses directly from the auditory cortex in both species in response to novel legal sequences or ones violating specific ordering relationships. Neural oscillations in both monkeys and humans in response to the nonsense word sequences show strikingly similar hierarchically nested low-frequency phase and high-gamma amplitude coupling, establishing this form of oscillatory coupling-previously associated with speech processing in the human auditory cortex-as an evolutionarily conserved biological process. Moreover, learned ordering relationships modulate the observed form of neural oscillatory coupling in both species, with temporally distinct neural oscillatory effects that appear to coordinate neuronal responses in the monkeys. This study identifies the conserved auditory cortical neural signatures involved in monitoring learned sequencing operations, evident as modulations of transient coupling and neuronal responses to temporally structured sensory input.

Evolution of the neural language network

The evolution of language correlates with distinct changes in the primate brain. The present article compares language-related brain regions and their white matter connectivity in the developing and mature human brain with the respective structures in the nonhuman primate brain. We will see that the functional specificity of the posterior portion of Broca's area (Brodmann area [BA 44]) and its dorsal fiber connection to the temporal cortex, shown to support the processing of structural hierarchy in humans, makes a crucial neural difference between the species. This neural circuit may thus be fundamental for the human syntactic capacity as the core of language.