What we mean when we say semantic: Toward a multidisciplinary semantic glossary

Tulving characterized semantic memory as a vast repository of meaning that underlies language and many other cognitive processes. This perspective on lexical and conceptual knowledge galvanized a new era of research undertaken by numerous fields, each with their own idiosyncratic methods and terminology. For example, "concept" has different meanings in philosophy, linguistics, and psychology. As such, many fundamental constructs used to delineate semantic theories remain underspecified and/or opaque. Weak construct specificity is among the leading causes of the replication crisis now facing psychology and related fields. Term ambiguity hinders cross-disciplinary communication, falsifiability, and incremental theory-building. Numerous cognitive subdisciplines (e.g., vision, affective neuroscience) have recently addressed these limitations via the development of consensus-based guidelines and definitions. The project to follow represents our effort to produce a multidisciplinary semantic glossary consisting of succinct definitions, background, principled dissenting views, ratings of agreement, and subjective confidence for 17 target constructs (e.g., abstractness, abstraction, concreteness, concept, embodied cognition, event semantics, lexical-semantic, modality, representation, semantic control, semantic feature, simulation, semantic distance, semantic dimension). We discuss potential benefits and pitfalls (e.g., implicit bias, prescriptiveness) of these efforts to specify a common nomenclature that other researchers might index in specifying their own theoretical perspectives (e.g., They said X, but I mean Y).

The Mind Hidden in Our Hands

Our hands are always with us and are used for communication all over the world. When children do not have an established language model to learn from, they use their hands to gesture, and these gestures take on the forms of language. In this role, the hands reveal the fundamental properties of the mind that give shape to language. When children do learn an established language, they again use their hands to gesture. These gestures do not look like language but form an integrated system with language. In this role, the hands can convey ideas not found in the language they accompany. In both contexts, gesture provides a clear view of the mind hidden in our hands.

A Pragmatic Turn in the Study of Early Executive Functions by Object Use and Gestures. A Case Study from 8 to 17 Months of Age at a Nursery School

The two first years of life are critical in the development of Executive Functions (EF). However, very little is known about their early manifestations, how they develop, how they relate to other psychological constructions or the status of other people's influence in this early development. The study of EFs has been carried out through standardised tasks, but some authors question their ecological validity and suggest an approach involving everyday situations and the challenges that children set for themselves. In this article we first review these issues in relation to the first manifestations of EFs. We secondly present a longitudinal case study at nursery school of a child between the ages of 8 and 17 months, considering the challenges and the means he employed in order to resolve them. We found that, from 8 months of age, the child gave himself challenges in relation to the functional uses of objects and instruments. He regulated his own behavior both through object and instrument uses and private gestures. He also involved the teacher at 17 months. This finding suggests that (1) the material world is particularly important in these early manifestations of EF, (2) teachers' interventions are essential. Implications of the findings for early years education are discussed.

Multiple Levels of Heuristic Reasoning Processes in Scientific Model Construction

Science historians have recognized the importance of heuristic reasoning strategies for constructing theories, but their extent and degree of organization are still poorly understood. This paper first consolidates a set of important heuristic strategies for constructing scientific models from three books, including studies in the history of genetics and electromagnetism, and an expert think-aloud study in the field of mechanics. The books focus on qualitative reasoning strategies (processes) involved in creative model construction, scientific breakthroughs, and conceptual change. Twenty four processes are examined, most of which are field-general, but all are heuristic in not being guaranteed to work. An organizing framework is then proposed as a four-level hierarchy of nested reasoning processes and subprocesses at different size and time scales, including: Level (L4) Several longer-time-scale Major Modeling Modes, such as Model Evolution and Model Competition; the former mode utilizes: (L3) Modeling Cycle Phases of Model Generation, Evaluation, and Modification under Constraints; which can utilize: (L2) Thirteen Tactical Heuristic Processes, e.g., Analogy, Infer new model feature (e.g., by running the model), etc.; many of which selectively utilize: (L1) Grounded Imagistic Processes, namely Mental Simulations and Structural Transformations. Incomplete serial ordering in the framework gives it an intermediate degree of organization that is neither anarchistic nor fully algorithmic. Its organizational structure is hypothesized to promote a difficult balance between divergent and convergent processes as it alternates between them in modeling cycles with increasingly constrained modifications. Videotaped think-aloud protocols that include depictive gestures and other imagery indicators indicate that the processes in L1 above can be imagistic. From neurological evidence that imagery uses many of the same brain regions as actual perception and action, it is argued that these expert reasoning processes are grounded in the sense of utilizing the perceptual and motor systems, and interconnections to and possible benefits for reasoning processes at higher levels are examined. The discussion examines whether this grounding and the various forms of organization in the framework may begin to explain how processes that are only sometimes useful and not guaranteed to work can combine successfully to achieve innovative scientific model construction.

Active learning: "Hands-on" meets "minds-on"

[Figure: see text].

Thinking Tools: Gestures Change Thought About Time

Our earliest tools are our bodies. Our hands raise and turn and toss and carry and push and pull, our legs walk and climb and kick allowing us to move and act in the world and to create the multitude of artifacts that improve our lives. The list of actions made by our hands and feet and other parts of our bodies is long. What is more remarkable is we turn those actions in the world into actions on thought through gestures, language, and graphics, thereby creating cognitive tools that expand the mind. The focus here is gesture; gestures transform actions on perceptible objects to actions on imagined thoughts, carrying meaning with them rapidly, precisely, and directly. We review evidence showing that gestures enhance our own thinking and change the thought of others. We illustrate the power of gestures in studies showing that gestures uniquely change conceptions of time, from sequential to simultaneous, from sequential to cyclical, and from a perspective embedded in a timeline to an external perspective looking on a timeline, and by so doing obviate the ambiguities of an embedded perspective. We draw parallels between representations in gesture and in graphics; both use marks or actions arrayed in space to communicate more immediately than symbolic language.

Inferences on enacted understanding: using immersive technologies to assess intuitive physical science knowledge

Purpose

This study aims to advance the proposal to use immersive virtual learning environments to stimulate and reveal deep-seated knowledge about science, giving instructors and researchers unique possibilities for assessing and identifying intuitive physical science knowledge. Aside from the ability to present rich and dynamic stimuli, these environments afford bodily enactment of people’s understanding, which draws less from declarative knowledge stores and more from everyday experiences with the physical world.

Design/methodology/approach

The authors ground their proposal in a critical review of the impact of stimulus and task characteristics of traditional physics inventories. Using a grounded theory approach, the authors present classifications and interpretations of observed bodily enactments of physics understandings in a study where participants enacted their understanding of force and motion of space in an immersive, interactive mixed reality (MR) environment.

Findings

The authors find that instances of these action categories can be interpreted as relating to underlying knowledge, often identified by other studies. The authors thus replicate a number of prior findings, which provide evidence to establish validation for using MR simulation as a tool for identifying people’s physical intuitions.

Research limitations/implications

This study targeted only a few specific physical science scenarios. Further, while a number of key insights about student knowledge came from the analysis, many of the observations are mere leads in need of further investigation and interpretation rather than core findings.

Originality/value

Immersive digital learning environments are primarily used for instruction. The authors propose to use and design them for assessment as well. This paper should prompt more research and development in this direction.

Limits on simulation approaches in intuitive physics

A popular explanation of the human ability for physical reasoning is that it depends on a sophisticated ability to perform mental simulations. According to this perspective, physical reasoning problems are approached by repeatedly simulating relevant aspects of a scenario, with noise, and making judgments based on aggregation over these simulations. In this paper, we describe three core tenets of simulation approaches, theoretical commitments that must be present in order for a simulation approach to be viable. The identification of these tenets threatens the plausibility of simulation as a theory of physical reasoning, because they appear to be incompatible with what we know about cognition more generally. To investigate this apparent contradiction, we describe three experiments involving simple physical judgments and predictions, and argue their results challenge these core predictions of theories of mental simulation.

Analyzing Self-Explanations in Mathematics: Gestures and Written Notes Do Matter

When learners self-explain, they try to make sense of new information. Although research has shown that bodily actions and written notes are an important part of learning, previous analyses of self-explanations rarely take into account written and non-verbal data produced spontaneously. In this paper, the extent to which interpretations of self-explanations are influenced by the systematic consideration of such data is investigated. The video recordings of 33 undergraduate students, who learned with worked-out examples dealing with complex numbers, were categorized successively including three different data bases: (a) verbal data, (b) verbal and written data, and (c) verbal, written and non-verbal data. Results reveal that including written data (notes) and non-verbal data (gestures and actions) leads to a more accurate analysis of self-explanations than an analysis solely based on verbal data. This influence is even stronger for the categorization of self-explanations as adequate or inadequate.

Modeling complexity: cognitive constraints and computational model-building in integrative systems biology

Modern integrative systems biology defines itself by the complexity of the problems it takes on through computational modeling and simulation. However in integrative systems biology computers do not solve problems alone. Problem solving depends as ever on human cognitive resources. Current philosophical accounts hint at their importance, but it remains to be understood what roles human cognition plays in computational modeling. In this paper we focus on practices through which modelers in systems biology use computational simulation and other tools to handle the cognitive complexity of their modeling problems so as to be able to make significant contributions to understanding, intervening in, and controlling complex biological systems. We thus show how cognition, especially processes of simulative mental modeling, is implicated centrally in processes of model-building. At the same time we suggest how the representational choices of what to model in systems biology are limited or constrained as a result. Such constraints help us both understand and rationalize the restricted form that problem solving takes in the field and why its results do not always measure up to expectations.

Spatial analogies pervade complex relational reasoning: Evidence from spontaneous gestures

How do people think about complex phenomena like the behavior of ecosystems? Here we hypothesize that people reason about such relational systems in part by creating spatial analogies, and we explore this possibility by examining spontaneous gestures. In two studies, participants read a written lesson describing positive and negative feedback systems and then explained the differences between them. Though the lesson was highly abstract and people were not instructed to gesture, people produced spatial gestures in abundance during their explanations. These gestures used space to represent simple abstract relations (e.g., increase) and sometimes more complex relational structures (e.g., negative feedback). Moreover, over the course of their explanations, participants’ gestures often cohered into larger analogical models of relational structure. Importantly, the spatial ideas evident in the hands were largely unaccompanied by spatial words. Gesture thus suggests that spatial analogies are pervasive in complex relational reasoning, even when language does not.

A Taxonomic Analysis of Abstraction

An important characteristic of knowledge is that it exists at multiple levels of abstraction. This article illustrates how different levels of abstraction influence perception, comprehension, categorization, memory, and thought. Theories exist for how abstraction influences each of these cognitive processes, but there are few unifying principles for discussing these theories within a common conceptual framework. My proposed taxonomy examines three senses of abstraction: (a) an abstract entity is a concept that has no material referent, (b) abstraction focuses on only some attributes of multicomponent stimuli, and (c) an abstract idea applies to many particular instances of a category. The first refers to instances, the second to attributes of instances, and the third to classes of instances. Concrete mental representations consist of modal images for instances, equivalent attributes, and exemplars or episodes for categories. Abstract mental representations consist of amodal propositions for instances, distinctive attributes, and rules or prototypes or schema for categories. I first apply the taxonomy to words, pictures, and problems. The next section shows how categorization strategies combine with abstraction at the attribute, instance, and category levels. The subsequent section applies the taxonomy to hierarchical (subordinate, basic, superordinate) levels. A concluding section proposes directions for further development.

From hands to minds: Gestures promote understanding

Gestures serve many roles in communication, learning and understanding both for those who view them and those who create them. Gestures are especially effective when they bear resemblance to the thought they represent, an advantage they have over words. Here, we examine the role of conceptually congruent gestures in deepening understanding of dynamic systems. Understanding the structure of dynamic systems is relatively easy, but understanding the actions of dynamic systems can be challenging. We found that seeing gestures representing actions enhanced understanding of the dynamics of a complex system as revealed in invented language, gestures and visual explanations. Gestures can map many meanings more directly than language, representing many concepts congruently. Designing and using gestures congruent with meaning can augment comprehension and learning.

Illuminating Mental Representations Through Speech and Gesture

Can the gestures people produce when describing algebra word problems offer insight into their mental representations of the problems? Twenty adults were asked to describe six word problems about constant change, and then to talk aloud as they solved the problems. Two problems depicted continuous change, two depicted discrete change, and two depicted change that could be construed as either continuous or discrete. Participants' verbal and gestured descriptions of the problems often incorporated information about manner of change. However, the information conveyed in gesture was not always the same as the information conveyed in speech. Participants' problem representations, as expressed in speech and gesture, were systematically related to their problem solutions. When gesture reinforced the representation expressed in the spoken description, participants were very likely to solve the problem using a strategy compatible with that representation—much more likely than when gesture did not reinforce the spoken description. The results indicate that gesture and speech together provide a better index of mental representation than speech alone.

From action to abstraction: Gesture as a mechanism of change

Piaget was a master at observing the routine behaviors children produce as they go from knowing less to knowing more about at a task, and making inferences not only about how the children understood the task at each point, but also about how they progressed from one point to the next. In this paper, I examine a routine behavior that Piaget overlooked-the spontaneous gestures speakers produce as they explain their solutions to a problem. These gestures are not mere hand waving. They reflect ideas that the speaker has about the problem, often ideas that are not found in that speaker's talk. But gesture can do more than reflect ideas-it can also change them. In this sense, gesture behaves like any other action; both gesture and action on objects facilitate learning problems on which training was given. However, only gesture promotes transferring the knowledge gained to problems that require generalization. Gesture is, in fact, a special kind of action in that it represents the world rather than directly manipulating the world (gesture does not move objects around). The mechanisms by which gesture and action promote learning may therefore differ-gesture is able to highlight components of an action that promote abstract learning while leaving out details that could tie learning to a specific context. Because it is both an action and a representation, gesture can serve as a bridge between the two and thus be a powerful tool for learning abstract ideas.

Particularities and universalities of the emergence of inductive generalization

Inductive generalization is the primary way by which human beings arrive at the construction of knowledge. Usually, it is assumed that it operates in a linear manner-each new feature becomes "piled up" in the inductive accumulation of evidence. We question this view, and otherwise claim that inductive generalization is essentially a non-linear dynamic process that fits the theoretical premises of the Dynamic Systems Theory. In our study, we explore the ability that young infants have when making inductive generalizations -previous studies show the existence of this capacity not earlier than at the age of 14 months. These studies have been cross-sectional in nature, but they do not offer an answer to the question of emergence of cognitive capabilities, therefore, a short-term longitudinal study is needed. Based on 3 case studies carried out longitudinally in infants ranging from 9 to 14 months, we demonstrate how the process of inductive generalization occurs from a conceptualization of nonlinear dynamic systems. We use Min - Max and State Space techniques, which allow us to show how the infant uses diverse pathways of actions with everyday objects to facilitate inductive generalization. The identified paths are not the same, they present differential and common moments that confirm the dynamic nature of development, and provide empirical evidence on the emergence of non-linear, non-sequential or inductive generalization.

From Gestures to Gaming: Visible Embodiment of Remote Actions

Teleoperation is the act of controlling an object that exists in a space, real or virtual, physically disconnected from the user. During such situations, it is not uncommon to observe those controlling the remote object exhibiting movement consistent with the behaviour of the remote object. Though this behaviour has no obvious impact on one's control of the remote object, it appears tied to one's intentions, thus, possibly representing an embodied representation of ongoing cognitive processes. In the present investigation, we applied a natural behaviour approach to test this notion, (a) first by identifying the representational basis for the behaviour and (b) by identifying factors that influence the occurrence of the behaviour. Each study involved observing participant behaviour while they played a racing video game. Results revealed that the spontaneous behaviour demonstrated in a teleoperation setting is tied to one's remote actions, rather than local actions or some combination of remote and local actions (Experiment 1). In addition, increasing task demand led to an increase in the occurrence of the spontaneous behaviour (Experiment 2). A third experiment was conducted to rule out the possible confound of greater immersion that tends to accompany greater demand (Experiment 3). The implications of these results not only suggest that spontaneous behaviour observed during teleoperation reflects a form of visible embodiment, sensitive to task demand, but also further emphasizes the utility of natural behaviour approaches for furthering our understanding of the relationship between the body and cognitive processes.

Rotating with rotated text: a natural behavior approach to investigating cognitive offloading

Determining how we use our body to support cognition represents an important part of understanding the embodied and embedded nature of cognition. In the present investigation, we pursue this question in the context of a common perceptual task. Specifically, we report a series of experiments investigating head tilt (i.e., external normalization) as a strategy in letter naming and reading stimuli that are upright or rotated. We demonstrate that the frequency of this natural behavior is modulated by the cost of stimulus rotation on performance. In addition, we demonstrate that external normalization can benefit performance. All of the results are consistent with the notion that external normalization represents a form of cognitive offloading and that effort is an important factor in the decision to adopt an internal or external strategy.

Individual differences in mental rotation: what does gesture tell us?

Gestures are common when people convey spatial information, for example, when they give directions or describe motion in space. Here, we examine the gestures speakers produce when they explain how they solved mental rotation problems (Shepard and Meltzer in Science 171:701-703, 1971). We asked whether speakers gesture differently while describing their problems as a function of their spatial abilities. We found that low-spatial individuals (as assessed by a standard paper-and-pencil measure) gestured more to explain their solutions than high-spatial individuals. While this finding may seem surprising, finer-grained analyses showed that low-spatial participants used gestures more often than high-spatial participants to convey "static only" information but less often than high-spatial participants to convey dynamic information. Furthermore, the groups differed in the types of gestures used to convey static information: high-spatial individuals were more likely than low-spatial individuals to use gestures that captured the internal structure of the block forms. Our gesture findings thus suggest that encoding block structure may be as important as rotating the blocks in mental spatial transformation.

"What if…": The Use of Conceptual Simulations in Scientific Reasoning

The term conceptual simulation refers to a type of everyday reasoning strategy commonly called "what if" reasoning. It has been suggested in a number of contexts that this type of reasoning plays an important role in scientific discovery; however, little direct evidence exists to support this claim. This article proposes that conceptual simulation is likely to be used in situations of informational uncertainty, and may be used to help scientists resolve that uncertainty. We conducted two studies to investigate the relationship between conceptual simulation and informational uncertainty. Study 1 was an in vivo study of expert scientists; the results suggest that scientists do use conceptual simulation in situations of informational uncertainty, and that they use conceptual simulation to make inferences from their data using the analogical reasoning process of alignment by similarity detection. Study 2 experimentally manipulated experts' level of uncertainty and provides further support for the hypothesis that conceptual simulation is more likely to be used in situations of informational uncertainty. Finally, we discuss the relationship between conceptual simulation and other types of reasoning using qualitative mental models.

A word in the hand: action, gesture and mental representation in humans and non-human primates

The movements we make with our hands both reflect our mental processes and help to shape them. Our actions and gestures can affect our mental representations of actions and objects. In this paper, we explore the relationship between action, gesture and thought in both humans and non-human primates and discuss its role in the evolution of language. Human gesture (specifically representational gesture) may provide a unique link between action and mental representation. It is kinaesthetically close to action and is, at the same time, symbolic. Non-human primates use gesture frequently to communicate, and do so flexibly. However, their gestures mainly resemble incomplete actions and lack the representational elements that characterize much of human gesture. Differences in the mirror neuron system provide a potential explanation for non-human primates' lack of representational gestures; the monkey mirror system does not respond to representational gestures, while the human system does. In humans, gesture grounds mental representation in action, but there is no evidence for this link in other primates. We argue that gesture played an important role in the transition to symbolic thought and language in human evolution, following a cognitive leap that allowed gesture to incorporate representational elements.

Spontaneous gestures influence strategy choices in problem solving

Do gestures merely reflect problem-solving processes, or do they play a functional role in problem solving? We hypothesized that gestures highlight and structure perceptual-motor information, and thereby make such information more likely to be used in problem solving. Participants in two experiments solved problems requiring the prediction of gear movement, either with gesture allowed or with gesture prohibited. Such problems can be correctly solved using either a perceptual-motor strategy (simulation of gear movements) or an abstract strategy (the parity strategy). Participants in the gesture-allowed condition were more likely to use perceptual-motor strategies than were participants in the gesture-prohibited condition. Gesture promoted use of perceptual-motor strategies both for participants who talked aloud while solving the problems (Experiment 1) and for participants who solved the problems silently (Experiment 2). Thus, spontaneous gestures influence strategy choices in problem solving.

Learning through gesture

When people talk, they move their hands-they gesture. Although these movements might appear to be meaningless hand waving, in fact they convey substantive information that is not always found in the accompanying speech. As a result, gesture can provide insight into thoughts that speakers have but do not know they have. Even more striking, gesture can mark a speaker as being in transition with respect to a task-learners who are on the verge of making progress on a task routinely produce gestures that convey information that is different from the information conveyed in speech. Gesture can thus be used to predict who will learn. In addition, evidence is mounting that gesture not only presages learning but also can play a role in bringing that learning about. Gesture can cause learning indirectly by influencing the learning environment or directly by influencing learners themselves. We can thus change our minds by moving our hands. WIREs Cogni Sci 2011 2 595-607 DOI: 10.1002/wcs.132 This article is categorized under: Psychology > Learning.

Gesture changes thought by grounding it in action

When people talk, they gesture. We show that gesture introduces action information into speakers' mental representations, which, in turn, affect subsequent performance. In Experiment 1, participants solved the Tower of Hanoi task (TOH1), explained (with gesture) how they solved it, and solved it again (TOH2). For all participants, the smallest disk in TOH1 was the lightest and could be lifted with one hand. For some participants (no-switch group), the disks in TOH2 were identical to those in TOH1. For others (switch group), the disk weights in TOH2 were reversed (so that the smallest disk was the heaviest and could not be lifted with one hand). The more the switch group's gestures depicted moving the smallest disk one-handed, the worse they performed on TOH2. This was not true for the no-switch group, nor for the switch group in Experiment 2, who skipped the explanation step and did not gesture. Gesturing grounds people's mental representations in action. When gestures are no longer compatible with the action constraints of a task, problem solving suffers.