The language of geometry: Fast comprehension of geometrical primitives and rules in human adults and preschoolers

Why concepts are (probably) vectors

For decades, cognitive scientists have debated what kind of representation might characterize human concepts. Whatever the format of the representation, it must allow for the computation of varied properties, including similarities, features, categories, definitions, and relations. It must also support the development of theories, ad hoc categories, and knowledge of procedures. Here, we discuss why vector-based representations provide a compelling account that can meet all these needs while being plausibly encoded into neural architectures. This view has become especially promising with recent advances in both large language models and vector symbolic architectures. These innovations show how vectors can handle many properties traditionally thought to be out of reach for neural models, including compositionality, definitions, structures, and symbolic computational processes.

Symbolic metaprogram search improves learning efficiency and explains rule learning in humans

Throughout their lives, humans seem to learn a variety of rules for things like applying category labels, following procedures, and explaining causal relationships. These rules are often algorithmically rich but are nonetheless acquired with minimal data and computation. Symbolic models based on program learning successfully explain rule-learning in many domains, but performance degrades quickly as program complexity increases. It remains unclear how to scale symbolic rule-learning methods to model human performance in challenging domains. Here we show that symbolic search over the space of metaprograms-programs that revise programs-dramatically improves learning efficiency. On a behavioral benchmark of 100 algorithmically rich rules, this approach fits human learning more accurately than alternative models while also using orders of magnitude less search. The computation required to match median human performance is consistent with conservative estimates of human thinking time. Our results suggest that metaprogram-like representations may help human learners to efficiently acquire rules.

Caricaturing Shapes in Visual Memory

When representing high-level stimuli, such as faces and animals, we tend to emphasize salient features-such as a face's prominent cheekbones or a bird's pointed beak. Such mental caricaturing leaves traces in memory, which exaggerates these distinctive qualities. How broadly does this phenomenon extend? Here, in six experiments (N = 700 adults), we explored how memory automatically caricatures basic units of visual processing-simple geometric shapes-even without task-related demands to do so. Participants saw a novel shape and then immediately adjusted a copy of that shape to match what they had seen. Surprisingly, participants reconstructed shapes in exaggerated form, amplifying curvature, enlarging salient parts, and so on. Follow-up experiments generalized this bias to new parameters, ruled out strategic responding, and amplified the effects in serial transmission. Thus, even the most basic stimuli we encounter are remembered as caricatures of themselves.

Compositional diversity in visual concept learning

Humans leverage compositionality to efficiently learn new concepts, understanding how familiar parts can combine together to form novel objects. In contrast, popular computer vision models struggle to make the same types of inferences, requiring more data and generalizing less flexibly than people do. Here, we study these distinctively human abilities across a range of different types of visual composition, examining how people classify and generate "alien figures" with rich relational structure. We also develop a Bayesian program induction model which searches for the best programs for generating the candidate visual figures, utilizing a large program space containing different compositional mechanisms and abstractions. In few shot classification tasks, we find that people and the program induction model can make a range of meaningful compositional generalizations, with the model providing a strong account of the experimental data as well as interpretable parameters that reveal human assumptions about the factors invariant to category membership (here, to rotation and changing part attachment). In few shot generation tasks, both people and the models are able to construct compelling novel examples, with people behaving in additional structured ways beyond the model capabilities, e.g. making choices that complete a set or reconfigure existing parts in new ways. To capture these additional behavioral patterns, we develop an alternative model based on neuro-symbolic program induction: this model also composes new concepts from existing parts yet, distinctively, it utilizes neural network modules to capture residual statistical structure. Together, our behavioral and computational findings show how people and models can produce a variety of compositional behavior when classifying and generating visual objects.

Neurocognitive Assessment of Mathematics-Related Capacities in Neurosurgical Patients

A precise neuropsychological assessment is of the utmost importance for neurosurgical patients undergoing the surgical excision of cerebral lesions. The assessment of mathematical abilities is usually limited to arithmetical operations while other fundamental visuo-spatial aspects closely linked to mathematics proficiency, such as the perception of numerical quantities and geometrical reasoning, are completely neglected. We evaluated these abilities with two objective and reproducible psychophysical tests, measuring numerosity perception and non-symbolic geometry, respectively. We tested sixteen neuro-oncological patients before the operation and six after the operation with classical neuropsychological tests and with two psychophysical tests. The scores of the classical neuropsychological tests were very heterogeneous, possibly due to the distinct location and histology of the tumors that might have spared (or not) brain areas subserving these abilities or allowed for plastic reorganization. Performance in the two non-symbolic tests reflected, on average, the presumed functional role of the lesioned areas, with participants with parietal and frontal lesions performing worse on these tests than patients with occipital and temporal lesions. Single-case analyses not only revealed some interesting exceptions to the group-level results (e.g., patients with parietal lesions performing well in the numerosity test), but also indicated that performance in the two tests was independent of non-verbal reasoning and visuo-spatial working memory. Our results highlight the importance of assessing non-symbolic numerical and geometrical abilities to complement typical neuropsychological batteries. However, they also suggest an avoidance of reliance on an excessively rigid localizationist approach when evaluating the neuropsychological profile of oncological patients.

The algorithmic origins of counting

The study of how children learn numbers has yielded one of the most productive research programs in cognitive development, spanning empirical and computational methods, as well as nativist and empiricist philosophies. This paper provides a tutorial on how to think computationally about learning models in a domain like number, where learners take finite data and go far beyond what they directly observe or perceive. To illustrate, this paper then outlines a model which acquires a counting procedure using observations of sets and words, extending the proposal of Piantadosi et al. (2012). This new version of the model responds to several critiques of the original work and outlines an approach which is likely appropriate for acquiring further aspects of mathematics.

Brain-imaging evidence for compression of binary sound sequences in human memory

According to the language-of-thought hypothesis, regular sequences are compressed in human memory using recursive loops akin to a mental program that predicts future items. We tested this theory by probing memory for 16-item sequences made of two sounds. We recorded brain activity with functional MRI and magneto-encephalography (MEG) while participants listened to a hierarchy of sequences of variable complexity, whose minimal description required transition probabilities, chunking, or nested structures. Occasional deviant sounds probed the participants' knowledge of the sequence. We predicted that task difficulty and brain activity would be proportional to the complexity derived from the minimal description length in our formal language. Furthermore, activity should increase with complexity for learned sequences, and decrease with complexity for deviants. These predictions were upheld in both fMRI and MEG, indicating that sequence predictions are highly dependent on sequence structure and become weaker and delayed as complexity increases. The proposed language recruited bilateral superior temporal, precentral, anterior intraparietal, and cerebellar cortices. These regions overlapped extensively with a localizer for mathematical calculation, and much less with spoken or written language processing. We propose that these areas collectively encode regular sequences as repetitions with variations and their recursive composition into nested structures.

Continual task learning in natural and artificial agents

How do humans and other animals learn new tasks? A wave of brain recording studies has investigated how neural representations change during task learning, with a focus on how tasks can be acquired and coded in ways that minimise mutual interference. We review recent work that has explored the geometry and dimensionality of neural task representations in neocortex, and computational models that have exploited these findings to understand how the brain may partition knowledge between tasks. We discuss how ideas from machine learning, including those that combine supervised and unsupervised learning, are helping neuroscientists understand how natural tasks are learned and coded in biological brains.

Are infants' preferences in the number change detection paradigm driven by sequence patterns?

Inter-individual differences in infants' numerosity processing have been assessed using a change detection paradigm, where participants were presented with two concurrent streams of images, one alternating between two numerosities and the other showing one constant numerosity. While most infants look longer at the changing stream in this paradigm, the reasons underlying these preferences have remained unclear. We suggest that, besides being attracted by numerosity changes, infants perhaps also respond to the alternating pattern of the changing stream. We conducted two experiments (N = 32) with 6-month-old infants to assess this hypothesis. In the first experiment, infants responded to changes in numerosity even when the changing stream showed numerosities in an unpredictable random order. In the second experiment, infants did not display any preference when an alternating stream was pitted against a random stream. These findings do not provide evidence that the alternating pattern of the changing stream contributes to drive infants' preferences. Instead, around the age of 6 months, infants' responses in the numerosity change detection paradigm appear to be mainly driven by changes in numerosity, with different levels of preference reflecting inter-individual difference in the acuity of numerosity perception.

The best game in town: The reemergence of the language-of-thought hypothesis across the cognitive sciences

Mental representations remain the central posits of psychology after many decades of scrutiny. However, there is no consensus about the representational format(s) of biological cognition. This paper provides a survey of evidence from computational cognitive psychology, perceptual psychology, developmental psychology, comparative psychology, and social psychology, and concludes that one type of format that routinely crops up is the language-of-thought (LoT). We outline six core properties of LoTs: (i) discrete constituents; (ii) role-filler independence; (iii) predicate-argument structure; (iv) logical operators; (v) inferential promiscuity; and (vi) abstract content. These properties cluster together throughout cognitive science. Bayesian computational modeling, compositional features of object perception, complex infant and animal reasoning, and automatic, intuitive cognition in adults all implicate LoT-like structures. Instead of regarding LoT as a relic of the previous century, researchers in cognitive science and philosophy-of-mind must take seriously the explanatory breadth of LoT-based architectures. We grant that the mind may harbor many formats and architectures, including iconic and associative structures as well as deep-neural-network-like architectures. However, as computational/representational approaches to the mind continue to advance, classical compositional symbolic structures - that is, LoTs - only prove more flexible and well-supported over time.

A language of thought for the mental representation of geometric shapes

In various cultures and at all spatial scales, humans produce a rich complexity of geometric shapes such as lines, circles or spirals. Here, we propose that humans possess a language of thought for geometric shapes that can produce line drawings as recursive combinations of a minimal set of geometric primitives. We present a programming language, similar to Logo, that combines discrete numbers and continuous integration to form higher-level structures based on repetition, concatenation and embedding, and we show that the simplest programs in this language generate the fundamental geometric shapes observed in human cultures. On the perceptual side, we propose that shape perception in humans involves searching for the shortest program that correctly draws the image (program induction). A consequence of this framework is that the mental difficulty of remembering a shape should depend on its minimum description length (MDL) in the proposed language. In two experiments, we show that encoding and processing of geometric shapes is well predicted by MDL. Furthermore, our hypotheses predict additive laws for the psychological complexity of repeated, concatenated or embedded shapes, which we confirm experimentally.

Symbols and mental programs: a hypothesis about human singularity

Natural language is often seen as the single factor that explains the cognitive singularity of the human species. Instead, we propose that humans possess multiple internal languages of thought, akin to computer languages, which encode and compress structures in various domains (mathematics, music, shape…). These languages rely on cortical circuits distinct from classical language areas. Each is characterized by: (i) the discretization of a domain using a small set of symbols, and (ii) their recursive composition into mental programs that encode nested repetitions with variations. In various tasks of elementary shape or sequence perception, minimum description length in the proposed languages captures human behavior and brain activity, whereas non-human primate data are captured by simpler nonsymbolic models. Our research argues in favor of discrete symbolic models of human thought.

Rational arbitration between statistics and rules in human sequence processing

Detecting and learning temporal regularities is essential to accurately predict the future. A long-standing debate in cognitive science concerns the existence in humans of a dissociation between two systems, one for handling statistical regularities governing the probabilities of individual items and their transitions, and another for handling deterministic rules. Here, to address this issue, we used finger tracking to continuously monitor the online build-up of evidence, confidence, false alarms and changes-of-mind during sequence processing. All these aspects of behaviour conformed tightly to a hierarchical Bayesian inference model with distinct hypothesis spaces for statistics and rules, yet linked by a single probabilistic currency. Alternative models based either on a single statistical mechanism or on two non-commensurable systems were rejected. Our results indicate that a hierarchical Bayesian inference mechanism, capable of operating over distinct hypothesis spaces for statistics and rules, underlies the human capability for sequence processing.

Working Memory for Spatial Sequences: Developmental and Evolutionary Factors in Encoding Ordinal and Relational Structures

Sequence learning is a ubiquitous facet of human and animal cognition. Here, using a common sequence reproduction task, we investigated whether and how the ordinal and relational structures linking consecutive elements are acquired by human adults, children, and macaque monkeys. While children and monkeys exhibited significantly lower precision than adults for spatial location and temporal order information, only monkeys appeared to exceedingly focus on the first item. Most importantly, only humans, regardless of age, spontaneously extracted the spatial relations between consecutive items and used a chunking strategy to compress sequences in working memory. Monkeys did not detect such relational structures, even after extensive training. Monkey behavior was captured by a conjunctive coding model, whereas a chunk-based conjunctive model explained more variance in humans. These age- and species-related differences are indicative of developmental and evolutionary mechanisms of sequence encoding and may provide novel insights into the uniquely human cognitive capacities.SIGNIFICANCE STATEMENT Sequence learning, the ability to encode the order of discrete elements and their relationships presented within a sequence, is a ubiquitous facet of cognition among humans and animals. By exploring sequence-processing abilities at different human developmental stages and in nonhuman primates, we found that only humans, regardless of age, spontaneously extracted the spatial relations between consecutive items and used an internal language to compress sequences in working memory. The findings provided insights into understanding the origins of sequence capabilities in humans and how they evolve through development to identify the unique aspects of human cognitive capacity, which includes the comprehension, learning, and production of sequences, and perhaps, above all, language processing.

One model for the learning of language

A major goal of linguistics and cognitive science is to understand what class of learning systems can acquire natural language. Until recently, the computational requirements of language have been used to argue that learning is impossible without a highly constrained hypothesis space. Here, we describe a learning system that is maximally unconstrained, operating over the space of all computations, and is able to acquire many of the key structures present in natural language from positive evidence alone. We demonstrate this by providing the same learning model with data from 74 distinct formal languages which have been argued to capture key features of language, have been studied in experimental work, or come from an interesting complexity class. The model is able to successfully induce the latent system generating the observed strings from small amounts of evidence in almost all cases, including for regular (e.g., an , [Formula: see text], and [Formula: see text]), context-free (e.g., [Formula: see text], and [Formula: see text]), and context-sensitive (e.g., [Formula: see text], and xx) languages, as well as for many languages studied in learning experiments. These results show that relatively small amounts of positive evidence can support learning of rich classes of generative computations over structures. The model provides an idealized learning setup upon which additional cognitive constraints and biases can be formalized.

Cerebral representation of sequence patterns across multiple presentation formats

The ability to detect the abstract pattern underlying a temporal sequence of events is crucial to many human activities, including language and mathematics, but its cortical correlates remain poorly understood. It is also unclear whether repeated exposure to the same sequence of sensory stimuli is sufficient to induce the encoding of an abstract amodal representation of the pattern. Using functional MRI, we probed the existence of such abstract codes for sequential patterns, their localization in the human brain, and their relation to existing language and math-responsive networks. We used a passive sequence violation paradigm, in which a given sequence is repeatedly presented before rare deviant sequences are introduced. We presented two binary patterns, AABB and ABAB, in four presentation formats, either visual or auditory, and either cued by the identity of the stimuli or by their spatial location. Regardless of the presentation format, a habituation to the repeated pattern and a response to pattern violations were seen in a set of inferior frontal, intraparietal and temporal areas. Within language areas, such pattern-violation responses were only found in the inferior frontal gyrus (IFG), whereas all math-responsive regions responded to pattern changes. Most of these regions also responded whenever the modality or the cue changed, suggesting a general sensitivity to violation detection. Thus, the representation of sequence patterns appears to be distributed, yet to include a core set of abstract amodal regions, particularly the IFG.

Geometry intuitions without vision? A study in blind children and adults

Geometry intuitions seem to be rooted in a non-verbal system that humans possess since early age. However, the mechanisms underlying the comprehension of basic geometric concepts remain elusive. Some authors have suggested that the starting point of geometry development could be found in the visual perception of specific features in our environment, thus conferring to vision a foundational role in the acquisition of geometric skills. To examine this assumption, a test probing intuitive understanding of basic geometric concepts was presented to congenitally blind children and adults. Participants had to detect the intruder among four different shapes, from which three instantiated a specific geometrical concept and one (the intruder) violated it. Although they performed above the chance level, the blind presented poorer performance than the sighted participants who did the task in the visual modality (i.e., with the eyes open), but performed equally well than the sighted who did the task in the tactile modality (i.e., with a blindfold). We therefore provide evidence that geometric abilities are impacted by the lack of vision.

Perception of geometric sequences and numerosity both predict formal geometric competence in primary school children

While most animals have a sense of number, only humans have developed symbolic systems to describe and organize mathematical knowledge. Some studies suggest that human arithmetical knowledge may be rooted in an ancient mechanism dedicated to perceiving numerosity, but it is not known if formal geometry also relies on basic, non-symbolic mechanisms. Here we show that primary-school children who spontaneously detect and predict geometrical sequences (non-symbolic geometry) perform better in school-based geometry tests indexing formal geometric knowledge. Interestingly, numerosity discrimination thresholds also predicted and explained a specific portion of variance of formal geometrical scores. The relation between these two non-symbolic systems and formal geometry was not explained by age or verbal reasoning skills. Overall, the results are in line with the hypothesis that some human-specific, symbolic systems are rooted in non-symbolic mechanisms.

Mental compression of spatial sequences in human working memory using numerical and geometrical primitives

How does the human brain store sequences of spatial locations? We propose that each sequence is internally compressed using an abstract, language-like code that captures its numerical and geometrical regularities. We exposed participants to spatial sequences of fixed length but variable regularity while their brain activity was recorded using magneto-encephalography. Using multivariate decoders, each successive location could be decoded from brain signals, and upcoming locations were anticipated prior to their actual onset. Crucially, sequences with lower complexity, defined as the minimal description length provided by the formal language, led to lower error rates and to increased anticipations. Furthermore, neural codes specific to the numerical and geometrical primitives of the postulated language could be detected, both in isolation and within the sequences. These results suggest that the human brain detects sequence regularities at multiple nested levels and uses them to compress long sequences in working memory.

Sensitivity to geometric shape regularity in humans and baboons: A putative signature of human singularity

Among primates, humans are special in their ability to create and manipulate highly elaborate structures of language, mathematics, and music. Here we show that this sensitivity to abstract structure is already present in a much simpler domain: the visual perception of regular geometric shapes such as squares, rectangles, and parallelograms. We asked human subjects to detect an intruder shape among six quadrilaterals. Although the intruder was always defined by an identical amount of displacement of a single vertex, the results revealed a geometric regularity effect: detection was considerably easier when either the base shape or the intruder was a regular figure comprising right angles, parallelism, or symmetry rather than a more irregular shape. This effect was replicated in several tasks and in all human populations tested, including uneducated Himba adults and French kindergartners. Baboons, however, showed no such geometric regularity effect, even after extensive training. Baboon behavior was captured by convolutional neural networks (CNNs), but neither CNNs nor a variational autoencoder captured the human geometric regularity effect. However, a symbolic model, based on exact properties of Euclidean geometry, closely fitted human behavior. Our results indicate that the human propensity for symbolic abstraction permeates even elementary shape perception. They suggest a putative signature of human singularity and provide a challenge for nonsymbolic models of human shape perception.

Counterexample Search in Diagram-Based Geometric Reasoning

Topological relations such as inside, outside, or intersection are ubiquitous to our spatial thinking. Here, we examined how people reason deductively with topological relations between points, lines, and circles in geometric diagrams. We hypothesized in particular that a counterexample search generally underlies this type of reasoning. We first verified that educated adults without specific math training were able to produce correct diagrammatic representations contained in the premisses of an inference. Our first experiment then revealed that subjects who correctly judged an inference as invalid almost always produced a counterexample to support their answer. Noticeably, even if the counterexample always bore a certain level of similarity to the initial diagram, we observed that an object was more likely to be varied between the two drawings if it was present in the conclusion of the inference. Experiments 2 and 3 then directly probed counterexample search. While participants were asked to evaluate a conclusion on the basis of a given diagram and some premisses, we modulated the difficulty of reaching a counterexample from the diagram. Our results indicate that both decreasing the counterexample density and increasing the counterexample distance impaired reasoning performance. Taken together, our results suggest that a search procedure for counterexamples, which proceeds object-wise, could underlie diagram-based geometric reasoning. Transposing points, lines, and circles to our spatial environment, the present study may ultimately provide insights on how humans reason about topological relations between positions, paths, and regions.

The computational origin of representation

Each of our theories of mental representation provides some insight into how the mind works. However, these insights often seem incompatible, as the debates between symbolic, dynamical, emergentist, sub-symbolic, and grounded approaches to cognition attest. Mental representations-whatever they are-must share many features with each of our theories of representation, and yet there are few hypotheses about how a synthesis could be possible. Here, I develop a theory of the underpinnings of symbolic cognition that shows how sub-symbolic dynamics may give rise to higher-level cognitive representations of structures, systems of knowledge, and algorithmic processes. This theory implements a version of conceptual role semantics by positing an internal universal representation language in which learners may create mental models to capture dynamics they observe in the world. The theory formalizes one account of how truly novel conceptual content may arise, allowing us to explain how even elementary logical and computational operations may be learned from a more primitive basis. I provide an implementation that learns to represent a variety of structures, including logic, number, kinship trees, regular languages, context-free languages, domains of theories like magnetism, dominance hierarchies, list structures, quantification, and computational primitives like repetition, reversal, and recursion. This account is based on simple discrete dynamical processes that could be implemented in a variety of different physical or biological systems. In particular, I describe how the required dynamics can be directly implemented in a connectionist framework. The resulting theory provides an "assembly language" for cognition, where high-level theories of symbolic computation can be implemented in simple dynamics that themselves could be encoded in biologically plausible systems.

A theory of memory for binary sequences: Evidence for a mental compression algorithm in humans

Working memory capacity can be improved by recoding the memorized information in a condensed form. Here, we tested the theory that human adults encode binary sequences of stimuli in memory using an abstract internal language and a recursive compression algorithm. The theory predicts that the psychological complexity of a given sequence should be proportional to the length of its shortest description in the proposed language, which can capture any nested pattern of repetitions and alternations using a limited number of instructions. Five experiments examine the capacity of the theory to predict human adults' memory for a variety of auditory and visual sequences. We probed memory using a sequence violation paradigm in which participants attempted to detect occasional violations in an otherwise fixed sequence. Both subjective complexity ratings and objective violation detection performance were well predicted by our theoretical measure of complexity, which simply reflects a weighted sum of the number of elementary instructions and digits in the shortest formula that captures the sequence in our language. While a simpler transition probability model, when tested as a single predictor in the statistical analyses, accounted for significant variance in the data, the goodness-of-fit with the data significantly improved when the language-based complexity measure was included in the statistical model, while the variance explained by the transition probability model largely decreased. Model comparison also showed that shortest description length in a recursive language provides a better fit than six alternative previously proposed models of sequence encoding. The data support the hypothesis that, beyond the extraction of statistical knowledge, human sequence coding relies on an internal compression using language-like nested structures.

The Child as Hacker

The scope of human learning and development poses a radical challenge for cognitive science. We propose that developmental theories can address this challenge by adopting perspectives from computer science. Many of our best models treat learning as analogous to computer programming because symbolic programs provide the most compelling account of sophisticated mental representations. We specifically propose that children's learning is analogous to a particular style of programming called hacking, making code better along many dimensions through an open-ended set of goals and activities. By contrast to existing theories, which depend primarily on local search and simple metrics, this view highlights the many features of good mental representations and the multiple complementary processes children use to create them.

Baboons (Papio papio) Process a Context-Free but Not a Context-Sensitive Grammar

Language processing involves the ability to master supra-regular grammars, that go beyond the level of complexity of regular grammars. This ability has been hypothesized to be a uniquely human capacity. Our study probed baboons' capacity to learn two supra-regular grammars of different levels of complexity: a context-free grammar generating sequences following a mirror structure (e.g., AB | BA, ABC | CBA) and a context-sensitive grammar generating sequences following a repeat structure (e.g., AB | AB, ABC | ABC), the latter requiring greater computational power to be processed. Fourteen baboons were tested in a prediction task, requiring them to track a moving target on a touchscreen. In distinct experiments, sequences of target locations followed one of the above two grammars, with rare violations. Baboons showed slower response times when violations occurred in mirror sequences, but did not react to violations in repeat sequences, suggesting that they learned the context-free (mirror) but not the context-sensitive (repeat) grammar. By contrast, humans tested with the same task learned both grammars. These data suggest a difference in sensitivity in baboons between a context-free and a context-sensitive grammar.

Towards a more flexible language of thought: Bayesian grammar updates after each concept exposure

Recent approaches to human concept learning have successfully combined the power of symbolic, infinitely productive rule systems and statistical learning to explain our ability to learn new concepts from just a few examples. The aim of most of these studies is to reveal the underlying language structuring these representations and providing a general substrate for thought. However, describing a model of thought that is fixed once trained is against the extensive literature that shows how experience shapes concept learning. Here, we ask about the plasticity of these symbolic descriptive languages. We perform a concept learning experiment that demonstrates that humans can change very rapidly the repertoire of symbols they use to identify concepts, by compiling expressions that are frequently used into new symbols of the language. The pattern of concept learning times is accurately described by a Bayesian agent that rationally updates the probability of compiling a new expression according to how useful it has been to compress concepts so far. By portraying the language of thought as a flexible system of rules, we also highlight the difficulties to pin it down empirically.

The evolution of early symbolic behavior in Homo sapiens

How did human symbolic behavior evolve? Dating up to about 100,000 y ago, the engraved ochre and ostrich eggshell fragments from the South African Blombos Cave and Diepkloof Rock Shelter provide a unique window into presumed early symbolic traditions of Homo sapiens and how they evolved over a period of more than 30,000 y. Using the engravings as stimuli, we report five experiments which suggest that the engravings evolved adaptively, becoming better-suited for human perception and cognition. More specifically, they became more salient, memorable, reproducible, and expressive of style and human intent. However, they did not become more discriminable over time between or within the two archeological sites. Our observations provide support for an account of the Blombos and Diepkloof engravings as decorations and as socially transmitted cultural traditions. By contrast, there was no clear indication that they served as denotational symbolic signs. Our findings have broad implications for our understanding of early symbolic communication and cognition in H. sapiens.

Beyond imitation: Zero-shot task transfer on robots by learning concepts as cognitive programs

Humans can infer concepts from image pairs and apply those in the physical world in a completely different setting, enabling tasks like IKEA assembly from diagrams. If robots could represent and infer high-level concepts, then it would notably improve their ability to understand our intent and to transfer tasks between different environments. To that end, we introduce a computational framework that replicates aspects of human concept learning. Concepts are represented as programs on a computer architecture consisting of a visual perception system, working memory, and action controller. The instruction set of this cognitive computer has commands for parsing a visual scene, directing gaze and attention, imagining new objects, manipulating the contents of a visual working memory, and controlling arm movement. Inferring a concept corresponds to inducing a program that can transform the input to the output. Some concepts require the use of imagination and recursion. Previously learned concepts simplify the learning of subsequent, more elaborate concepts and create a hierarchy of abstractions. We demonstrate how a robot can use these abstractions to interpret novel concepts presented to it as schematic images and then apply those concepts in very different situations. By bringing cognitive science ideas on mental imagery, perceptual symbols, embodied cognition, and deictic mechanisms into the realm of machine learning, our work brings us closer to the goal of building robots that have interpretable representations and common sense.

Representation of spatial sequences using nested rules in human prefrontal cortex

Memory for spatial sequences does not depend solely on the number of locations to be stored, but also on the presence of spatial regularities. Here, we show that the human brain quickly stores spatial sequences by detecting geometrical regularities at multiple time scales and encoding them in a format akin to a programming language. We measured gaze-anticipation behavior while spatial sequences of variable regularity were repeated. Participants' behavior suggested that they quickly discovered the most compact description of each sequence in a language comprising nested rules, and used these rules to compress the sequence in memory and predict the next items. Activity in dorsal inferior prefrontal cortex correlated with the amount of compression, while right dorsolateral prefrontal cortex encoded the presence of embedded structures. Sequence learning was accompanied by a progressive differentiation of multi-voxel activity patterns in these regions. We propose that humans are endowed with a simple "language of geometry" which recruits a dorsal prefrontal circuit for geometrical rules, distinct from but close to areas involved in natural language processing.

The statistical shape of geometric reasoning

Geometric reasoning has an inherent dissonance: its abstract axioms and propositions refer to perfect, idealized entities, whereas its use in the physical world relies on dynamic perception of objects. How do abstract Euclidean concepts, dynamics, and statistics come together to support our intuitive geometric reasoning? Here, we address this question using a simple geometric task – planar triangle completion. An analysis of the distribution of participants’ errors in localizing a fragmented triangle’s missing corner reveals scale-dependent deviations from a deterministic Euclidean representation of planar triangles. By considering the statistical physics of the process characterized via a correlated random walk with a natural length scale, we explain these results and further predict participants’ estimates of the missing angle, measured in a second task. Our model also predicts the results of a categorical reasoning task about changes in the triangle size and shape even when such completion strategies need not be invoked. Taken together, our findings suggest a critical role for noisy physical processes in our reasoning about elementary Euclidean geometry.

Cortical circuits for mathematical knowledge: evidence for a major subdivision within the brain's semantic networks

Is mathematical language similar to natural language? Are language areas used by mathematicians when they do mathematics? And does the brain comprise a generic semantic system that stores mathematical knowledge alongside knowledge of history, geography or famous people? Here, we refute those views by reviewing three functional MRI studies of the representation and manipulation of high-level mathematical knowledge in professional mathematicians. The results reveal that brain activity during professional mathematical reflection spares perisylvian language-related brain regions as well as temporal lobe areas classically involved in general semantic knowledge. Instead, mathematical reflection recycles bilateral intraparietal and ventral temporal regions involved in elementary number sense. Even simple fact retrieval, such as remembering that 'the sine function is periodical' or that 'London buses are red', activates dissociated areas for math versus non-math knowledge. Together with other fMRI and recent intracranial studies, our results indicated a major separation between two brain networks for mathematical and non-mathematical semantics, which goes a long way to explain a variety of facts in neuroimaging, neuropsychology and developmental disorders.This article is part of a discussion meeting issue 'The origins of numerical abilities'.

Bayesian validation of grammar productions for the language of thought

Probabilistic proposals of Language of Thoughts (LoTs) can explain learning across different domains as statistical inference over a compositionally structured hypothesis space. While frameworks may differ on how a LoT may be implemented computationally, they all share the property that they are built from a set of atomic symbols and rules by which these symbols can be combined. In this work we propose an extra validation step for the set of atomic productions defined by the experimenter. It starts by expanding the defined LoT grammar for the cognitive domain with a broader set of arbitrary productions and then uses Bayesian inference to prune the productions from the experimental data. The result allows the researcher to validate that the resulting grammar still matches the intuitive grammar chosen for the domain. We then test this method in the language of geometry, a specific LoT model for geometrical sequence learning. Finally, despite the fact of the geometrical LoT not being a universal (i.e. Turing-complete) language, we show an empirical relation between a sequence’s probability and its complexity consistent with the theoretical relationship for universal languages described by Levin’s Coding Theorem.

Production of Supra-regular Spatial Sequences by Macaque Monkeys

Understanding and producing embedded sequences in language, music, or mathematics, is a central characteristic of our species. These domains are hypothesized to involve a human-specific competence for supra-regular grammars, which can generate embedded sequences that go beyond the regular sequences engendered by finite-state automata. However, is this capacity truly unique to humans? Using a production task, we show that macaque monkeys can be trained to produce time-symmetrical embedded spatial sequences whose formal description requires supra-regular grammars or, equivalently, a push-down stack automaton. Monkeys spontaneously generalized the learned grammar to novel sequences, including longer ones, and could generate hierarchical sequences formed by an embedding of two levels of abstract rules. Compared to monkeys, however, preschool children learned the grammars much faster using a chunking strategy. While supra-regular grammars are accessible to nonhuman primates through extensive training, human uniqueness may lie in the speed and learning strategy with which they are acquired.

Neurophysiological dynamics of phrase-structure building during sentence processing

Although sentences unfold sequentially, one word at a time, most linguistic theories propose that their underlying syntactic structure involves a tree of nested phrases rather than a linear sequence of words. Whether and how the brain builds such structures, however, remains largely unknown. Here, we used human intracranial recordings and visual word-by-word presentation of sentences and word lists to investigate how left-hemispheric brain activity varies during the formation of phrase structures. In a broad set of language-related areas, comprising multiple superior temporal and inferior frontal sites, high-gamma power increased with each successive word in a sentence but decreased suddenly whenever words could be merged into a phrase. Regression analyses showed that each additional word or multiword phrase contributed a similar amount of additional brain activity, providing evidence for a merge operation that applies equally to linguistic objects of arbitrary complexity. More superficial models of language, based solely on sequential transition probability over lexical and syntactic categories, only captured activity in the posterior middle temporal gyrus. Formal model comparison indicated that the model of multiword phrase construction provided a better fit than probability-based models at most sites in superior temporal and inferior frontal cortices. Activity in those regions was consistent with a neural implementation of a bottom-up or left-corner parser of the incoming language stream. Our results provide initial intracranial evidence for the neurophysiological reality of the merge operation postulated by linguists and suggest that the brain compresses syntactically well-formed sequences of words into a hierarchy of nested phrases.