Gesturing during mental problem solving reduces eye movements, especially for individuals with lower visual working memory capacity

Gestures can help children learn mathematics: how researchers can work with teachers to make gesture studies applicable to classrooms

The gestures we produce serve a variety of functions-they affect our communication, guide our attention and help us think and change the way we think. Gestures can consequently also help us learn, generalize what we learn and retain that knowledge over time. The effects of gesture-based instruction in mathematics have been well studied. However, few of these studies are directly applicable to classroom environments. Here, we review literature that highlights the benefits of producing and observing gestures when teaching and learning mathematics, and we provide suggestions for designing research studies with an eye towards how gestures can feasibly be applied to classroom learning. This article is part of the theme issue 'Minds in movement: embodied cognition in the age of artificial intelligence'.

SMG: A Micro-gesture Dataset Towards Spontaneous Body Gestures for Emotional Stress State Analysis

We explore using body gestures for hidden emotional state analysis. As an important non-verbal communicative fashion, human body gestures are capable of conveying emotional information during social communication. In previous works, efforts have been made mainly on facial expressions, speech, or expressive body gestures to interpret classical expressive emotions. Differently, we focus on a specific group of body gestures, called micro-gestures (MGs), used in the psychology research field to interpret inner human feelings. MGs are subtle and spontaneous body movements that are proven, together with micro-expressions, to be more reliable than normal facial expressions for conveying hidden emotional information. In this work, a comprehensive study of MGs is presented from the computer vision aspect, including a novel spontaneous micro-gesture (SMG) dataset with two emotional stress states and a comprehensive statistical analysis indicating the correlations between MGs and emotional states. Novel frameworks are further presented together with various state-of-the-art methods as benchmarks for automatic classification, online recognition of MGs, and emotional stress state recognition. The dataset and methods presented could inspire a new way of utilizing body gestures for human emotion understanding and bring a new direction to the emotion AI community. The source code and dataset are made available: https://github.com/mikecheninoulu/SMG.

Integrating Embodied Cognition and Information Processing: A Combined Model of the Role of Gesture in Children's Mathematical Environments

Children learn and use various strategies to solve math problems. One way children's math learning can be supported is through their use of and exposure to hand gestures. Children's self-produced gestures can reveal unique, math-relevant knowledge that is not contained in their speech. Additionally, these gestures can assist with their math learning and problem solving by supporting their cognitive processes, such as executive function. The gestures that children observe during math instructions are also linked to supporting cognition. Specifically, children are better able to learn, retain, and generalize knowledge about math when that information is presented within the gestures that accompany an instructor's speech. To date, no conceptual model provides an outline regarding how these gestures and the math environment are connected, nor how they may interact with children's underlying cognitive capacities such as their executive function. In this review, we propose a new model based on an integration of the information processing approach and theory of embodied cognition. We provide an in-depth review of the related literature and consider how prior research aligns with each link within the proposed model. Finally, we discuss the utility of the proposed model as it pertains to future research endeavors.

Gesture Use and Processing: A Review on Individual Differences in Cognitive Resources

Speakers use spontaneous hand gestures as they speak and think. These gestures serve many functions for speakers who produce them as well as for listeners who observe them. To date, studies in the gesture literature mostly focused on group-comparisons or the external sources of variation to examine when people use, process, and benefit from using and observing gestures. However, there are also internal sources of variation in gesture use and processing. People differ in how frequently they use gestures, how salient their gestures are, for what purposes they produce gestures, and how much they benefit from using and seeing gestures during comprehension and learning depending on their cognitive dispositions. This review addresses how individual differences in different cognitive skills relate to how people employ gestures in production and comprehension across different ages (from infancy through adulthood to healthy aging) from a functionalist perspective. We conclude that speakers and listeners can use gestures as a compensation tool during communication and thinking that interacts with individuals' cognitive dispositions.

Does Gesture Lighten the Load? The Case of Verbal Analogies

Gesturing has been shown to relay benefits to speakers and listeners alike. Speakers, for instance, may be able to reduce their working memory load through gesture. Studies with children and adults have demonstrated that gesturing while describing how to solve a problem can help to save cognitive resources related to that explanation, allowing them to be allocated to a secondary task. The majority of research in this area focuses on procedural mathematical problem solving; however, the present study examines how gesture interacts with working memory load during a verbal reasoning task: verbal analogies. Unlike previous findings which report improved performance on secondary tasks while gesturing during a primary task, our results show that participants showed better performance in a secondary memory task when being prohibited from gesturing during their explanation of verbal analogies compared to being allowed to gesture. These results suggest that the relationship between gesture and working memory may be more nuanced, with the type of task and gestures produced influencing how gestures interact with working memory load.

The role of gesture as simulated action in reinterpretation of mental imagery

In two experiments, we examined the role of gesture in reinterpreting a mental image. In Experiment 1, we found that participants gestured more about a figure they had learned through manual exploration than about a figure they had learned through vision. This supports claims that gestures emerge from the activation of perception-relevant actions during mental imagery. In Experiment 2, we investigated whether such gestures have a causal role in affecting the quality of mental imagery. Participants were randomly assigned to gesture, not gesture, or engage in a manual interference task as they attempted to reinterpret a figure they had learned through manual exploration. We found that manual interference significantly impaired participants' success on the task. Taken together, these results suggest that gestures reflect mental imaginings of interactions with a mental image and that these imaginings are critically important for mental manipulation and reinterpretation of that image. However, our results suggest that enacting the imagined movements in gesture is not critically important on this particular task.

A Narrative Review of School-Based Physical Activity for Enhancing Cognition and Learning: The Importance of Relevancy and Integration

Engaging in regular physical activity can have substantial cognitive and academic benefits for children, and is generally promoted for its beneficial effects on children's physical and mental health. Although embodied cognition research has convincingly shown the integral relationship of the human body and mind, in schools physical activity and cognitive activity are typically treated as unrelated processes. Consequently, most physical activities used are neither sufficiently relevant for nor fully integrated into the learning tasks. In reviewing the literature regarding the integration of physical activity into education to promote cognition and learning, two main lines of research emerged: exercise and cognition research vs. embodied cognition research. In this narrative review, we describe these two separately evolved schools of thought, highlighting their differences and commonalities. In categorising the existing studies on a 2 × 2 matrix, concerning the two main categories of relevance for and integration into the learning task, it becomes clear where the different foci lie, and how both lines of research could profit from learning from each other. Finally, a new instructional model that integrates task-relevant physical activities into the cognitive/learning task is proposed to inform both further research and educational practice.

Co-thought gesturing supports more complex problem solving in subjects with lower visual working-memory capacity

During silent problem solving, hand gestures arise that have no communicative intent. The role of such co-thought gestures in cognition has been understudied in cognitive research as compared to co-speech gestures. We investigated whether gesticulation during silent problem solving supported subsequent performance in a Tower of Hanoi problem-solving task, in relation to visual working-memory capacity and task complexity. Seventy-six participants were assigned to either an instructed gesture condition or a condition that allowed them to gesture, but without explicit instructions to do so. This resulted in three gesture groups: (1) non-gesturing; (2) spontaneous gesturing; (3) instructed gesturing. In line with the embedded/extended cognition perspective on gesture, gesturing benefited complex problem-solving performance for participants with a lower visual working-memory capacity, but not for participants with a lower spatial working-memory capacity.

Measuring Cognitive Load in Embodied Learning Settings

In recent years, research on embodied cognition has inspired a number of studies on multimedia learning and instructional psychology. However, in contrast to traditional research on education and multimedia learning, studies on embodied learning (i.e., focusing on bodily action and perception in the context of education) in some cases pose new problems for the measurement of cognitive load. This review provides an overview over recent studies on embodied learning in which cognitive load was measured using surveys, behavioral data, or physiological measures. The different methods are assessed in terms of their success in finding differences of cognitive load in embodied learning scenarios. At the same time, we highlight the most important challenges for researchers aiming to include these measures into their study designs. The main issues we identified are: (1) Subjective measures must be appropriately phrased to be useful for embodied learning; (2) recent findings indicate potentials as well as problematic aspects of dual-task measures; (3) the use of physiological measures offers great potential, but may require mobile equipment in the context of embodied scenarios; (4) meta-cognitive measures can be useful extensions of cognitive load measurement for embodied learning.