How Language Programs the Mind

Why concepts are (probably) vectors

For decades, cognitive scientists have debated what kind of representation might characterize human concepts. Whatever the format of the representation, it must allow for the computation of varied properties, including similarities, features, categories, definitions, and relations. It must also support the development of theories, ad hoc categories, and knowledge of procedures. Here, we discuss why vector-based representations provide a compelling account that can meet all these needs while being plausibly encoded into neural architectures. This view has become especially promising with recent advances in both large language models and vector symbolic architectures. These innovations show how vectors can handle many properties traditionally thought to be out of reach for neural models, including compositionality, definitions, structures, and symbolic computational processes.

Group Coordination Catalyzes Individual and Cultural Intelligence

A large program of research has aimed to ground large-scale cultural phenomena in processes taking place within individual minds. For example, investigating whether individual agents equipped with the right social learning strategies can enable cumulative cultural evolution given long enough time horizons. However, this approach often omits the critical group-level processes that mediate between individual agents and multi-generational societies. Here, we argue that interacting groups are a necessary and explanatory level of analysis, linking individual and collective intelligence through two characteristic feedback loops. In the first loop, more sophisticated individual-level social learning mechanisms based on Theory of Mind facilitate group-level complementarity, allowing distributed knowledge to be compositionally recombined in groups; these group-level innovations, in turn, ease the cognitive load on individuals. In the second loop, societal-level processes of cumulative culture provide groups with new cognitive technologies, including shared language and conceptual abstractions, which set in motion new group-level processes to further coordinate, recombine, and innovate. Taken together, these cycles establish group-level interaction as a dual engine of intelligence, catalyzing both individual cognition and cumulative culture.

Symbolic metaprogram search improves learning efficiency and explains rule learning in humans

Throughout their lives, humans seem to learn a variety of rules for things like applying category labels, following procedures, and explaining causal relationships. These rules are often algorithmically rich but are nonetheless acquired with minimal data and computation. Symbolic models based on program learning successfully explain rule-learning in many domains, but performance degrades quickly as program complexity increases. It remains unclear how to scale symbolic rule-learning methods to model human performance in challenging domains. Here we show that symbolic search over the space of metaprograms-programs that revise programs-dramatically improves learning efficiency. On a behavioral benchmark of 100 algorithmically rich rules, this approach fits human learning more accurately than alternative models while also using orders of magnitude less search. The computation required to match median human performance is consistent with conservative estimates of human thinking time. Our results suggest that metaprogram-like representations may help human learners to efficiently acquire rules.

The effects of internal representations on performance and fluidity in a motor task

Individuals can differ in the mode in which they experience conscious thought. These differences in visualisation and verbalisation can also be evident during motor control. The Internal Representation Questionnaire (IRQ) was developed to measure propensity to engage certain types of representations, but its ability to predict motor control and links to reinvestment and motor imagery have not been tested. 159 included participants completed the IRQ, movement specific reinvestment scale (MSRS), and a novel online motor task before and after a period of practice. Results showed that the IRQ Verbal and Orthographic factors were significant predictors of scores on the MSRS. The IRQ factor of Manipulational Representations predicted motor performance both before and after practice. The fluidity of executed movements were predicted by the IRQ verbalisation factor where higher propensity to verbalise was associated with higher levels of jitter, but only after a period of practice. Results suggest there may be some informative conceptual overlap between internal verbalisations and reinvestment and that the propensity to manipulate internal representations may be predictive of motor performance in new tasks. The IRQ has potential to be a valuable tool for predicting motor performance.

Compositional diversity in visual concept learning

Humans leverage compositionality to efficiently learn new concepts, understanding how familiar parts can combine together to form novel objects. In contrast, popular computer vision models struggle to make the same types of inferences, requiring more data and generalizing less flexibly than people do. Here, we study these distinctively human abilities across a range of different types of visual composition, examining how people classify and generate "alien figures" with rich relational structure. We also develop a Bayesian program induction model which searches for the best programs for generating the candidate visual figures, utilizing a large program space containing different compositional mechanisms and abstractions. In few shot classification tasks, we find that people and the program induction model can make a range of meaningful compositional generalizations, with the model providing a strong account of the experimental data as well as interpretable parameters that reveal human assumptions about the factors invariant to category membership (here, to rotation and changing part attachment). In few shot generation tasks, both people and the models are able to construct compelling novel examples, with people behaving in additional structured ways beyond the model capabilities, e.g. making choices that complete a set or reconfigure existing parts in new ways. To capture these additional behavioral patterns, we develop an alternative model based on neuro-symbolic program induction: this model also composes new concepts from existing parts yet, distinctively, it utilizes neural network modules to capture residual statistical structure. Together, our behavioral and computational findings show how people and models can produce a variety of compositional behavior when classifying and generating visual objects.

Is language-of-thought the best game in the town we live?

There are towns in which language-of-thought (LoT) is the best game. But do we live in one? I go through three properties that characterize the LoT hypothesis: Discrete constituents, role-filler independence, and logical operators, and argue that in each case predictions from the LoT hypothesis are a poor fit to actual human cognition. As a hypothesis of what human cognition ought to be like, LoT departs from empirical reality.

Interactions of lexical and conceptual representations: Evidence from EEG

We examined whether meanings automatically activate linguistic forms, and whether these forms affect semantic decisions. Participants were presented sequentially with pairs of pictures and decided whether the objects in the pictures were related. At no point did they name the pictures. The object names of the experimental stimuli were ambiguous either in orthography (homographs), phonology (homophones), or both (homonyms), or unambiguous. We show that the lexical characteristics of the name of the objects affect a semantic decision about real world relations, in an online measure (N400), in addition to offline behavioral measures. We show a dissociation between conceptual and lexical recognition, where an earlier component (N230), was affected by relatedness, but was not sensitive to the lexical characteristics. We interpret this as supporting the hypothesis that semantic recognition occurs before the automatic lexical activation of the object name, but that once linguistic representations are activated, they affect semantic integration.

The debate over understanding in AI's large language models

We survey a current, heated debate in the artificial intelligence (AI) research community on whether large pretrained language models can be said to understand language-and the physical and social situations language encodes-in any humanlike sense. We describe arguments that have been made for and against such understanding and key questions for the broader sciences of intelligence that have arisen in light of these arguments. We contend that an extended science of intelligence can be developed that will provide insight into distinct modes of understanding, their strengths and limitations, and the challenge of integrating diverse forms of cognition.

Editorial concepts in interaction: social engagement and inner experiences

This theme issue aims to view the literature on concepts through a novel lens, that of social interaction and its influence on inner experiences. It discusses unsolved problems in literature on concepts, emphasizing the distinction between concrete versus abstract concepts and external versus internal grounding. This introductory article reflects the two research streams that the theme aims to bridge-in this area, the dimension of embodied interaction with others and how this influences the interaction with ourselves is still underexplored. In the first part, we discuss recent trends in social cognition, showing how interacting with others influences our concepts. In the second part, we address how social interactions become part of our inner world in a Vygotskian fashion. First, we illustrate how interoception, emotion and metacognition are connected with concepts and knowledge. Second, we deal with how language, in both its outer and inner form, can empower cognition and concepts. We also briefly describe how novel experimental and computational methods contribute to investigating the online use of concepts. Overall, this introductory article outlines the potentialities of an integrated and interactive approach that can give new, fresh life to a topic, that of concepts, which lies at the root of human cognition. This article is part of the theme issue 'Concepts in interaction: social engagement and inner experiences'.

Rethinking the role of language in embodied cognition

There has been a lot of recent interest in the way that language might enhance embodied cognition. This interest is driven in large part by a growing body of evidence implicating the language system in various aspects of semantic memory-including, but not limited to, its apparent contribution to abstract concepts. In this essay, I develop and defend a novel account of the cognitive role played by language in our concepts. This account relies on the embodied nature of the language system itself, diverges in significant ways from traditional accounts, and is part of a flexible, multimodal and multilevel view of our conceptual system. This article is part of the theme issue 'Concepts in interaction: social engagement and inner experiences'.

The language marker hypothesis

According to the language marker hypothesis language has provided homo sapiens with a rich symbolic system that plays a central role in interpreting signals delivered by our sensory apparatus, in shaping action goals, and in creating a powerful tool for reasoning and inferencing. This view provides an important correction on embodied accounts of language that reduce language to action, perception, emotion and mental simulation. The presence of a language system has, however, also important consequences for perception, action, emotion, and memory. Language stamps signals from perception, action, and emotional systems with rich cognitive markers that transform the role of these signals in the overall cognitive architecture of the human mind. This view does not deny that language is implemented by means of universal principles of neural organization. However, language creates the possibility to generate rich internal models of the world that are shaped and made accessible by the characteristics of a language system. This makes us less dependent on direct action-perception couplings and might even sometimes go at the expense of the veridicality of perception. In cognitive (neuro)science the pendulum has swung from language as the key to understand the organization of the human mind to the perspective that it is a byproduct of perception and action. It is time that it partly swings back again.

Semantic projection recovers rich human knowledge of multiple object features from word embeddings

How is knowledge about word meaning represented in the mental lexicon? Current computational models infer word meanings from lexical co-occurrence patterns. They learn to represent words as vectors in a multidimensional space, wherein words that are used in more similar linguistic contexts-that is, are more semantically related-are located closer together. However, whereas inter-word proximity captures only overall relatedness, human judgements are highly context dependent. For example, dolphins and alligators are similar in size but differ in dangerousness. Here, we use a domain-general method to extract context-dependent relationships from word embeddings: 'semantic projection' of word-vectors onto lines that represent features such as size (the line connecting the words 'small' and 'big') or danger ('safe' to 'dangerous'), analogous to 'mental scales'. This method recovers human judgements across various object categories and properties. Thus, the geometry of word embeddings explicitly represents a wealth of context-dependent world knowledge.

Ultimate Grounding of Abstract Concepts: A Graded Account

Abstraction, one of the hallmarks of human cognition, continues to be the topic of a strong debate. The primary disagreement concerns whether or not abstract concepts can be accounted for within the scope of embodied cognition. In this paper, we introduce the embodied approach to conceptual knowledge and distinguish between embodiment and grounding, where grounding is the general term for how concepts initially acquire their meaning. Referring to numerous pieces of empirical evidence, we emphasise that, ultimately, all concepts are acquired via interaction with the world via two main pathways: embodiment and social interaction. The first pathway is direct and primarily involves action/perception, interoception and emotions. The second pathway is indirect, being mediated by language in particular. Evidence from neuroscience, psychology and cognitive linguistics shows these pathways have different properties, roles in cognition and temporal profiles. Human development also places revealing constraints on how children develop the ability to reason more abstractly as they grow up. We recognize language as a crucial cognitive faculty with several roles enabling the acquisition of abstract concepts indirectly. Three detailed case studies on body-specificity hypothesis, abstract verbs and mathematics are used to argue that a compelling case has accumulated in favour of the ultimate grounding of abstract concepts in an agent’s interaction with its world, primarily relying on the direct pathway. We consolidate the debate through multidisciplinary evidence for the idea that abstractness is a graded, rather than a binary property of concepts.

Extensional Superposition and Its Relation to Compositionality in Language and Thought

Semantic composition in language must be closely related to semantic composition in thought. But the way the two processes are explained differs considerably. Focusing primarily on propositional content, language theorists generally take semantic composition to be a truth-conditional process. Focusing more on extensional content, cognitive theorists take it to be a form of concept combination. But though deep, this disconnect is not irreconcilable. Both areas of theory assume that extensional (i.e., denotational) meanings must play a role. As this article demonstrates, they also have the potential to fulfill a mediative function. What is shown is that extensional meanings are themselves inherently compositional. On this basis, it becomes possible to model semantic composition without assuming the existence of any specifically linguistic/conceptual apparatus. Examples are presented to demonstrate this direct style of modeling. Abstract connections between composition in thought and language can then be made, raising the prospect of a more unified, theoretical account of semantic composition.

Analogy as a Catalyst for Cumulative Cultural Evolution

Analogies, broadly defined, map novel concepts onto familiar concepts, making them essential for perception, reasoning, and communication. We argue that analogy-building served a critical role in the evolution of cumulative culture by allowing humans to learn and transmit complex behavioural sequences that would otherwise be too cognitively demanding or opaque to acquire. The emergence of a protolanguage consisting of simple labels would have provided early humans with the cognitive tools to build explicit analogies and to communicate them to others. This focus on analogy-building can shed new light on the coevolution of cognition and culture and addresses recent calls for better integration of the field of cultural evolution with cognitive science.

Assessing abstract thought and its relation to language with a new nonverbal paradigm: Evidence from aphasia

In recent years, language has been shown to play a number of important cognitive roles over and above the communication of thoughts. One hypothesis gaining support is that language facilitates thought about abstract categories, such as democracy or prediction. To test this proposal, a novel set of semantic memory task trials, designed for assessing abstract thought non-linguistically, were normed for levels of abstractness. The trials were rated as more or less abstract to the degree that answering them required the participant to abstract away from both perceptual features and common setting associations corresponding to the target image. The normed materials were then used with a population of people with aphasia to assess the relationship of abstract thought to language. While the language-impaired group with aphasia showed lower overall accuracy and longer response times than controls in general, of special note is that their response times were significantly longer as a function of a trial's degree of abstractness. Further, the aphasia group's response times in reporting their degree of confidence (a separate, metacognitive measure) were negatively correlated with their language production abilities, with lower language scores predicting longer metacognitive response times. These results provide some support for the hypothesis that language is an important aid to abstract thought and to metacognition about abstract thought.

The Child as Hacker

The scope of human learning and development poses a radical challenge for cognitive science. We propose that developmental theories can address this challenge by adopting perspectives from computer science. Many of our best models treat learning as analogous to computer programming because symbolic programs provide the most compelling account of sophisticated mental representations. We specifically propose that children's learning is analogous to a particular style of programming called hacking, making code better along many dimensions through an open-ended set of goals and activities. By contrast to existing theories, which depend primarily on local search and simple metrics, this view highlights the many features of good mental representations and the multiple complementary processes children use to create them.

Inner speech

Inner speech travels under many aliases: the inner voice, verbal thought, thinking in words, internal verbalization, "talking in your head," the "little voice in the head," and so on. It is both a familiar element of first-person experience and a psychological phenomenon whose complex cognitive components and distributed neural bases are increasingly well understood. There is evidence that inner speech plays a variety of cognitive roles, from enabling abstract thought, to supporting metacognition, memory, and executive function. One active area of controversy concerns the relation of inner speech to auditory verbal hallucinations (AVHs) in schizophrenia, with a common proposal being that sufferers of AVH misidentify their own inner speech as being generated by someone else. Recently, researchers have used artificial intelligence to translate the neural and neuromuscular signatures of inner speech into corresponding outer speech signals, laying the groundwork for a variety of new applications and interventions. This article is categorized under: Philosophy > Foundations of Cognitive Science Linguistics > Language in Mind and Brain Philosophy > Consciousness Philosophy > Psychological Capacities.

The Role of Inner Speech in Executive Functioning Tasks: Schizophrenia With Auditory Verbal Hallucinations and Autistic Spectrum Conditions as Case Studies

Several theories propose that one of the core functions of inner speech (IS) is to support subjects in the completion of cognitively effortful tasks, especially those involving executive functions (EF). In this paper we focus on two populations who notoriously encounter difficulties in performing EF tasks, namely, people diagnosed with schizophrenia who experience auditory verbal hallucinations (Sz-AVH) and people with autism spectrum conditions (ASC). We focus on these two populations because they represent two different ways in which IS can fail to help in EF tasks, which can be illustrative for other mental conditions. First, we review the main components of EF (see section "Executive Functions"). Then we explain the functions that IS is taken to perform in the domain of EF (see section "Inner Speech and Executive Functions") and review the evidence concerning problems about EF in the two populations of our study: Sz-AVH (see section "Executive Functions and Inner Speech in Sz-AVH") and ASC (see section "Executive Function and Inner Speech in ASC"). After this we further detail our account about what a properly functioning IS can do for both populations and how different IS profiles may impact EF performance: in the case of Sz-AVH, the uncontrolled and intrusive character of IS negatively affects EF performance, whereas in ASC, EF is not sufficiently supported by IS, given the tendency in this population to present a diminished use of IS (see section "IS in ASC and Sz-AVH: How It Relates to EF"). We finally briefly discuss Attention Deficit/Hyperactivity Disorder (ADHD) and Developmental Language Disorders (DLD) (see section "Further Considerations").

How Abstract (Non-embodied) Linguistic Representations Augment Cognitive Control

Recent scholarship emphasizes the scaffolding role of language for cognition. Language, it is claimed, is a cognition-enhancing niche (Clark, 2006), a programming tool for cognition (Lupyan and Bergen, 2016), even neuroenhancement (Dove, 2019) and augments cognitive functions such as memory, categorization, cognitive control, and meta-cognitive abilities ("thinking about thinking"). Yet, the notion that language enhances or augments cognition, and in particular, cognitive control does not easily fit in with embodied approaches to language processing, or so we will argue. Accounts aiming to explain how language enhances various cognitive functions often employ a notion of abstract representation. Yet, embodied approaches to language processing have it that language processing crucially, according to some accounts even exclusively, involves embodied, modality-specific, i.e., non-abstract representations. In coming to understand a particular phrase or sentence, a prior experience has to be simulated or reenacted. The representation thus activated is embodied (modality-specific) as sensorimotor regions of the brain are thereby recruited. In this paper, we will first discuss the notion of representation, clarify what it takes for a representation to be embodied or abstract, and distinguish between conceptual and (other) linguistic representations. We will then put forward a characterization of cognitive control and examine its representational infrastructure. The remainder of the paper will be devoted to arguing that language augments cognitive control. To that end, we will draw on two lines of research, which investigate how language augments cognitive control: (i) research on the availability of linguistic labels and (ii) research on the active usage of a linguistic code, specifically, in inner speech. Eventually, we will argue that the cognition-enhancing capacity of language can be explained once we assume that it provides us with (a) abstract, non-embodied representations and with (b) abstract, sparse linguistic representations that may serve as easy-to-manipulate placeholders for fully embodied or otherwise more detailed representations.

Interactions between language, thought, and perception: Cognitive and neural perspectives

ABSTRACT The role that language plays in shaping non-linguistic cognitive and perceptual systems has been the subject of much theoretical and experimental attention over the past half-century. Understanding how language interacts with non-linguistic systems can provide insight into broader constraints on cognitive and brain organization. The papers that form this volume investigate various ways in which linguistic structure can interact with and influence how speakers think about and perceive the world, and the related issue of the constraints that in turn shape linguistic representations. These theoretical and empirical contributions support deeper understanding of the interactions between language, thought, and perception, and motivate new approaches for developing directional predictions at both the neural and cognitive levels.

The Internal Representations Questionnaire: Measuring modes of thinking

Does the format in which we experience our moment-to-moment thoughts vary from person to person? Many people claim that their thinking takes place in an inner voice and that using language outside of interpersonal communication is a regular experience for them. Other people disagree. We present a novel measure, the Internal Representation Questionnaire (IRQ) designed to assess people's subjective mode of internal representations, and to quantify individual differences in "modes of thinking" along multiple factors in a single questionnaire. Exploratory factor analysis identified four factors: Internal Verbalization, Visual Imagery, Orthographic Imagery, and Representational Manipulation. All four factors were positively correlated with one another, but accounted for unique predictions. We describe the properties of the IRQ and report a test of its ability to predict patterns of interference in a speeded word-picture verification task. Taken together, the results suggest that self-reported differences in how people internally represent their thoughts relates to differences in processing familiar images and written words.

More than a scaffold: Language is a neuroenhancement

What role does language play in our thoughts? A longstanding proposal that has gained traction among supporters of embodied or grounded cognition suggests that it serves as a cognitive scaffold. This idea turns on the fact that language-with its ability to capture statistical regularities, leverage culturally acquired information, and engage grounded metaphors-is an effective and readily available support for our thinking. In this essay, I argue that language should be viewed as more than this; it should be viewed as a neuroenhancement. The neurologically realized language system is an important subcomponent of a flexible, multimodal, and multilevel conceptual system. It is not merely a source for information about the world but also a computational add-on that extends our conceptual reach. This approach provides a compelling explanation of the course of development, our facility with abstract concepts, and even the scope of language-specific influences on cognition.

Language as a disruptive technology: abstract concepts, embodiment and the flexible mind

A growing body of evidence suggests that cognition is embodied and grounded. Abstract concepts, though, remain a significant theoretical challenge. A number of researchers have proposed that language makes an important contribution to our capacity to acquire and employ concepts, particularly abstract ones. In this essay, I critically examine this suggestion and ultimately defend a version of it. I argue that a successful account of how language augments cognition should emphasize its symbolic properties and incorporate a view of embodiment that recognizes the flexible, multimodal and task-related nature of action, emotion and perception systems. On this view, language is an ontogenetically disruptive cognitive technology that expands our conceptual reach.This article is part of the theme issue 'Varieties of abstract concepts: development, use and representation in the brain'.

Language is more abstract than you think, or, why aren't languages more iconic?

How abstract is language? We show that abstractness pervades every corner of language, going far beyond the usual examples of freedom and justice In the light of the ubiquity of abstract words, the need to understand where abstract meanings come from becomes ever more acute. We argue that the best source of knowledge about abstract meanings may be language itself. We then consider a seemingly unrelated question: Why isn't language more iconic? Iconicity-a resemblance between the form of words and their meanings-can be immensely useful in language learning and communication. Languages could be much more iconic than they currently are. So why aren't they? We suggest that one reason is that iconicity is inimical to abstraction because iconic forms are too connected to specific contexts and sensory depictions. Form-meaning arbitrariness may allow language to better convey abstract meanings.This article is part of the theme issue 'Varieties of abstract concepts: development, use and representation in the brain'.

Is color an integral part of a rich mental simulation?

Research suggests that language comprehenders simulate visual features such as shape during language comprehension. In sentence-picture verification tasks, whenever pictures match the shape or orientation implied by the previous sentence, responses are faster than when the pictures mismatch implied visual aspects. However, mixed results have been demonstrated when the sentence-picture paradigm was applied to color (Connell, Cognition, 102(3), 476–485, 2007; Zwaan & Pecher, PLOS ONE, 7(12), e51382, 2012). One of the aims of the current investigation was to resolve this issue. This was accomplished by conceptually replicating the original study on color, using the same paradigm but a different stimulus set. The second goal of this study was to assess how much perceptual information is included in a mental simulation. We examined this by reducing color saturation, a manipulation that does not sacrifice object identifiability. If reduction of one aspect of color does not alter the match effect, it would suggest that not all perceptual information is relevant for a mental simulation. Our results did not support this: We found a match advantage when objects were shown at normal levels of saturation, but this match advantage disappeared when saturation was reduced, yet still aided in object recognition compared to when color was entirely removed. Taken together, these results clearly show a strong match effect for color, and the perceptual richness of mental simulations during language comprehension.

The influence of communication mode on written language processing and beyond

Empirical evidence suggests a broad impact of communication mode on cognition at large, beyond language processing. Using a sign language since infancy might shape the representation of words and other linguistic stimuli - for example, incorporating in it the movements and signs used to express them. Once integrated into linguistic representations, this visuo-motor content can affect deaf signers' linguistic and cognitive processing.

New Frontiers in Language Evolution and Development

This article introduces the Special Issue and its focus on research in language evolution with emphasis on theory as well as computational and robotic modeling. A key theme is based on the growth of evolutionary developmental biology or evo-devo. The Special Issue consists of 13 articles organized in two sections: A) Theoretical foundations and B) Modeling and simulation studies. All the papers are interdisciplinary in nature, encompassing work in biological and linguistic foundations for the study of language evolution as well as a variety of computational and robotic modeling efforts shedding light on how language may be developed and may have evolved.

Constructing nonhuman animal emotion

Scientists and lay-people alike have long been fascinated with the emotional lives of nonhuman animals. To date, scientific approaches to the study of 'animal' emotion have assumed that emotions are biologically evolutionarily conserved, hardwired and have discrete behavioral and physiological outputs. According to this view, emotions and their outputs are homologous across species, allowing humans to accurately perceive (or 'read') animal emotion using our own concepts of what emotions are. In this paper, I discuss the challenges to that perspective and propose using an alternative theoretical approach to understand animal emotion. Adopting this alternative approach, which represents a collection of similar theories (referred to as 'Theories of Constructed Emotion'), changes the questions that we ask about animal emotion, how we study emotion across phylogeny and advance translational science, and how we understand the evolution of emotion.

Building machines that learn and think like people

Recent progress in artificial intelligence has renewed interest in building systems that learn and think like people. Many advances have come from using deep neural networks trained end-to-end in tasks such as object recognition, video games, and board games, achieving performance that equals or even beats that of humans in some respects. Despite their biological inspiration and performance achievements, these systems differ from human intelligence in crucial ways. We review progress in cognitive science suggesting that truly human-like learning and thinking machines will have to reach beyond current engineering trends in both what they learn and how they learn it. Specifically, we argue that these machines should (1) build causal models of the world that support explanation and understanding, rather than merely solving pattern recognition problems; (2) ground learning in intuitive theories of physics and psychology to support and enrich the knowledge that is learned; and (3) harness compositionality and learning-to-learn to rapidly acquire and generalize knowledge to new tasks and situations. We suggest concrete challenges and promising routes toward these goals that can combine the strengths of recent neural network advances with more structured cognitive models.

Grounding Abstractness: Abstract Concepts and the Activation of the Mouth

One key issue for theories of cognition is how abstract concepts, such as freedom, are represented. According to the WAT (Words As social Tools) proposal, abstract concepts activate both sensorimotor and linguistic/social information, and their acquisition modality involves the linguistic experience more than the acquisition of concrete concepts. We report an experiment in which participants were presented with abstract and concrete definitions followed by concrete and abstract target-words. When the definition and the word matched, participants were required to press a key, either with the hand or with the mouth. Response times and accuracy were recorded. As predicted, we found that abstract definitions and abstract words yielded slower responses and more errors compared to concrete definitions and concrete words. More crucially, there was an interaction between the target-words and the effector used to respond (hand, mouth). While responses with the mouth were overall slower, the advantage of the hand over the mouth responses was more marked with concrete than with abstract concepts. The results are in keeping with grounded and embodied theories of cognition and support the WAT proposal, according to which abstract concepts evoke linguistic-social information, hence activate the mouth. The mechanisms underlying the mouth activation with abstract concepts (re-enactment of acquisition experience, or re-explanation of the word meaning, possibly through inner talk) are discussed. To our knowledge this is the first behavioral study demonstrating with real words that the advantage of the hand over the mouth is more marked with concrete than with abstract concepts, likely because of the activation of linguistic information with abstract concepts.

Why Are There Different Languages? The Role of Adaptation in Linguistic Diversity

Why are there different languages? A common explanation is that different languages arise from the gradual accumulation of random changes. Here, we argue that, beyond these random factors, linguistic differences, from sounds to grammars, may also reflect adaptations to different environments in which the languages are learned and used. The aspects of the environment that could shape language include the social, the physical, and the technological.

Words Jump-Start Vision: A Label Advantage in Object Recognition

People use language to shape each other's behavior in highly flexible ways. Effects of language are often assumed to be "high-level" in that, whereas language clearly influences reasoning, decision making, and memory, it does not influence low-level visual processes. Here, we test the prediction that words are able to provide top-down guidance at the very earliest stages of visual processing by acting as powerful categorical cues. We investigated whether visual processing of images of familiar animals and artifacts was enhanced after hearing their name (e.g., "dog") compared with hearing an equally familiar and unambiguous nonverbal sound (e.g., a dog bark) in 14 English monolingual speakers. Because the relationship between words and their referents is categorical, we expected words to deploy more effective categorical templates, allowing for more rapid visual recognition. By recording EEGs, we were able to determine whether this label advantage stemmed from changes to early visual processing or later semantic decision processes. The results showed that hearing a word affected early visual processes and that this modulation was specific to the named category. An analysis of ERPs showed that the P1 was larger when people were cued by labels compared with equally informative nonverbal cues-an enhancement occurring within 100 ms of image onset, which also predicted behavioral responses occurring almost 500 ms later. Hearing labels modulated the P1 such that it distinguished between target and nontarget images, showing that words rapidly guide early visual processing.