1
|
Zappa A, Bolger D, Pergandi JM, Fargier R, Mestre D, Frenck-Mestre C. The Neural Correlates of Embodied L2 Learning: Does Embodied L2 Verb Learning Affect Representation and Retention? NEUROBIOLOGY OF LANGUAGE (CAMBRIDGE, MASS.) 2024; 5:360-384. [PMID: 38911460 PMCID: PMC11192445 DOI: 10.1162/nol_a_00132] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 08/10/2020] [Accepted: 12/18/2023] [Indexed: 06/25/2024]
Abstract
We investigated how naturalistic actions in a highly immersive, multimodal, interactive 3D virtual reality (VR) environment may enhance word encoding by recording EEG in a pre/post-test learning paradigm. While behavior data have shown that coupling word encoding with gestures congruent with word meaning enhances learning, the neural underpinnings of this effect have yet to be elucidated. We coupled EEG recording with VR to examine whether embodied learning improves learning and creates linguistic representations that produce greater motor resonance. Participants learned action verbs in an L2 in two different conditions: specific action (observing and performing congruent actions on virtual objects) and pointing (observing actions and pointing to virtual objects). Pre- and post-training participants performed a match-mismatch task as we measured EEG (variation in the N400 response as a function of match between observed actions and auditory verbs) and a passive listening task while we measured motor activation (mu [8-13 Hz] and beta band [13-30 Hz] desynchronization during auditory verb processing) during verb processing. Contrary to our expectations, post-training results revealed neither semantic nor motor effects in either group when considered independently of learning success. Behavioral results showed a great deal of variability in learning success. When considering performance, low performance learners showed no semantic effect and high performance learners exhibited an N400 effect for mismatch versus match trials post-training, independent of the type of learning. Taken as a whole, our results suggest that embodied processes can play an important role in L2 learning.
Collapse
Affiliation(s)
- Ana Zappa
- Institute of Neurosciences and Department of Cognition, Development and Educational Psychology at University of Barcelona, Barcelona, Spain
- Cognition and Brain Plasticity Unit, Bellvitge Biomedical Research Institute, L’Hospitalet de Llobregat, Barcelona, Spain
| | - Deidre Bolger
- Aix Marseille Univ, CNRS, LPL, Aix-en-Provence, France
| | | | | | | | | |
Collapse
|
2
|
Raghavan R, Raviv L, Peeters D. What's your point? Insights from virtual reality on the relation between intention and action in the production of pointing gestures. Cognition 2023; 240:105581. [PMID: 37573692 DOI: 10.1016/j.cognition.2023.105581] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/14/2022] [Revised: 07/03/2023] [Accepted: 07/26/2023] [Indexed: 08/15/2023]
Abstract
Human communication involves the process of translating intentions into communicative actions. But how exactly do our intentions surface in the visible communicative behavior we display? Here we focus on pointing gestures, a fundamental building block of everyday communication, and investigate whether and how different types of underlying intent modulate the kinematics of the pointing hand and the brain activity preceding the gestural movement. In a dynamic virtual reality environment, participants pointed at a referent to either share attention with their addressee, inform their addressee, or get their addressee to perform an action. Behaviorally, it was observed that these different underlying intentions modulated how long participants kept their arm and finger still, both prior to starting the movement and when keeping their pointing hand in apex position. In early planning stages, a neurophysiological distinction was observed between a gesture that is used to share attitudes and knowledge with another person versus a gesture that mainly uses that person as a means to perform an action. Together, these findings suggest that our intentions influence our actions from the earliest neurophysiological planning stages to the kinematic endpoint of the movement itself.
Collapse
Affiliation(s)
- Renuka Raghavan
- Max Planck Institute for Psycholinguistics, Nijmegen, The Netherlands; Radboud University, Donders Institute for Brain, Cognition, and Behavior, Nijmegen, The Netherlands
| | - Limor Raviv
- Max Planck Institute for Psycholinguistics, Nijmegen, The Netherlands; Centre for Social, Cognitive and Affective Neuroscience (cSCAN), University of Glasgow, United Kingdom
| | - David Peeters
- Tilburg University, Department of Communication and Cognition, TiCC, Tilburg, The Netherlands.
| |
Collapse
|
3
|
Bechtold L, Cosper SH, Malyshevskaya A, Montefinese M, Morucci P, Niccolai V, Repetto C, Zappa A, Shtyrov Y. Brain Signatures of Embodied Semantics and Language: A Consensus Paper. J Cogn 2023; 6:61. [PMID: 37841669 PMCID: PMC10573703 DOI: 10.5334/joc.237] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/06/2022] [Accepted: 07/29/2022] [Indexed: 10/17/2023] Open
Abstract
According to embodied theories (including embodied, embedded, extended, enacted, situated, and grounded approaches to cognition), language representation is intrinsically linked to our interactions with the world around us, which is reflected in specific brain signatures during language processing and learning. Moving on from the original rivalry of embodied vs. amodal theories, this consensus paper addresses a series of carefully selected questions that aim at determining when and how rather than whether motor and perceptual processes are involved in language processes. We cover a wide range of research areas, from the neurophysiological signatures of embodied semantics, e.g., event-related potentials and fields as well as neural oscillations, to semantic processing and semantic priming effects on concrete and abstract words, to first and second language learning and, finally, the use of virtual reality for examining embodied semantics. Our common aim is to better understand the role of motor and perceptual processes in language representation as indexed by language comprehension and learning. We come to the consensus that, based on seminal research conducted in the field, future directions now call for enhancing the external validity of findings by acknowledging the multimodality, multidimensionality, flexibility and idiosyncrasy of embodied and situated language and semantic processes.
Collapse
Affiliation(s)
- Laura Bechtold
- Institute for Experimental Psychology, Department for Biological Psychology, Heinrich-Heine University Düsseldorf, Germany
| | - Samuel H. Cosper
- Institute of Cognitive Science, University of Osnabrück, Germany
| | - Anastasia Malyshevskaya
- Centre for Cognition and Decision making, Institute for Cognitive Neuroscience, HSE University, Russian Federation
- Potsdam Embodied Cognition Group, Cognitive Sciences, University of Potsdam, Germany
| | | | | | - Valentina Niccolai
- Institute of Clinical Neuroscience and Medical Psychology, Medical Faculty, Heinrich-Heine University Düsseldorf, Germany
| | - Claudia Repetto
- Department of Psychology, Università Cattolica del Sacro Cuore, Milan, Italy
| | - Ana Zappa
- Laboratoire parole et langage, Aix-Marseille Université, Aix-en-Provence, France
| | - Yury Shtyrov
- Centre for Cognition and Decision making, Institute for Cognitive Neuroscience, HSE University, Russian Federation
- Center of Functionally Integrative Neuroscience, Department of Clinical Medicine, Aarhus University, Denmark
| |
Collapse
|
4
|
O’Shea H. Mapping relational links between motor imagery, action observation, action-related language, and action execution. Front Hum Neurosci 2022; 16:984053. [DOI: 10.3389/fnhum.2022.984053] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/01/2022] [Accepted: 10/07/2022] [Indexed: 11/06/2022] Open
Abstract
Actions can be physically executed, observed, imagined, or simply thought about. Unifying mental processes, such as simulation, emulation, or predictive processing, are thought to underlie different action types, whether they are mental states, as in the case of motor imagery and action observation, or involve physical execution. While overlapping brain activity is typically observed across different actions which indicates commonalities, research interest is also concerned with investigating the distinct functional components of these action types. Unfortunately, untangling subtleties associated with the neurocognitive bases of different action types is a complex endeavour due to the high dimensional nature of their neural substrate (e.g., any action process is likely to activate multiple brain regions thereby having multiple dimensions to consider when comparing across them). This has impeded progress in action-related theorising and application. The present study addresses this challenge by using the novel approach of multidimensional modeling to reduce the high-dimensional neural substrate of four action-related behaviours (motor imagery, action observation, action-related language, and action execution), find the least number of dimensions that distinguish or relate these action types, and characterise their neurocognitive relational links. Data for the model comprised brain activations for action types from whole-brain analyses reported in 53 published articles. Eighty-two dimensions (i.e., 82 brain regions) for the action types were reduced to a three-dimensional model, that mapped action types in ordination space where the greater the distance between the action types, the more dissimilar they are. A series of one-way ANOVAs and post-hoc comparisons performed on the mean coordinates for each action type in the model showed that across all action types, action execution and concurrent action observation (AO)-motor imagery (MI) were most neurocognitively similar, while action execution and AO were most dissimilar. Most action types were similar on at least one neurocognitive dimension, the exception to this being action-related language. The import of the findings are discussed in terms of future research and implications for application.
Collapse
|
5
|
Bower IS, Hill AT, Enticott PG. Functional brain connectivity during exposure to the scale and color of interior built environments. Hum Brain Mapp 2022; 44:447-457. [PMID: 36053213 PMCID: PMC9842925 DOI: 10.1002/hbm.26061] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/24/2022] [Revised: 07/29/2022] [Accepted: 08/07/2022] [Indexed: 01/25/2023] Open
Abstract
Understanding brain activity linked to built environment exposure is important, as it may affect underlying cognitive, perceptual, and emotional processes, which have a critical influence in our daily life. As our time spent inside buildings is rising, and mental health problems have become more prevalent, it is important we investigate how design characteristics of the built environment impact brain function. In this study, we utilized electroencephalography to understand whether the design elements of scale and color of interior built environments modulate functional brain connectivity (i.e., brain network communication). Using a Cave Automatic Virtual Environment, while controlling indoor environmental quality responsible for physiological comfort, healthy adult participants aged 18-55 years (66 for scale, subset of 18 for color), were exposed to context-neutral indoor room scenes presented for two-minutes each. Our results show that both enlarging and reducing scale enhanced theta connectivity across the left temporoparietal region and right frontal region. We also found when reducing the built environment scale, there was a network exhibiting greater high-gamma connectivity, over the right frontoparietal region. For color, the condition (blue) contrasted to our achromatic control (white) increased theta connectivity in the frontal hemispheres. These findings identify a link between theta and gamma oscillations during exposure to the scale and color of the built environment, showing that design characteristics of the built environment could affect our cognitive processes and mental health. This suggests that, through the design of buildings, we may be able to mediate performance and health outcomes, which could lead to major health and economic benefits for society.
Collapse
Affiliation(s)
- Isabella S. Bower
- Cognitive Neuroscience Unit, School of Psychology, Faculty of HealthDeakin UniversityGeelongVictoriaAustralia,School of Architecture and Built Environment, Faculty of Science, Engineering and Built EnvironmentDeakin UniversityGeelongVictoriaAustralia
| | - Aron T. Hill
- Cognitive Neuroscience Unit, School of Psychology, Faculty of HealthDeakin UniversityGeelongVictoriaAustralia
| | - Peter G. Enticott
- Cognitive Neuroscience Unit, School of Psychology, Faculty of HealthDeakin UniversityGeelongVictoriaAustralia
| |
Collapse
|
6
|
Li X, Luo D, Wang C, Xia Y, Jin H. Motor features of abstract verbs determine their representations in the motor system. Front Psychol 2022; 13:957426. [PMID: 36110272 PMCID: PMC9469731 DOI: 10.3389/fpsyg.2022.957426] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/31/2022] [Accepted: 07/19/2022] [Indexed: 11/13/2022] Open
Abstract
Embodied cognition theory posits that concept representations inherently rely on sensorimotor experiences that accompany their acquisitions. This is well established through concrete concepts. However, it is debatable whether representations of abstract concepts are based on sensorimotor representations. This study investigated the causal role of associated motor experiences that accompany concept acquisition in the involvement of the motor system in the abstract verb processing. Through two experiments, we examined the action–sentence compatibility effect, in the test phase after an increase in motor features during the learning phase for abstract verbs with low motor features (Experiment 1) or novel words with no conceptual features at all (Experiment 2). After associated motor experiences were added in the word learning phase, action–sentence compatibility effect was found in the semantic processing tasks during the test phase for abstract verbs (Experiment 1a) and novel words (Experiment 2). This was lacking in the word font color judgment task requiring no semantic processing (Experiment 1b). Coupled with our previous study, these findings suggest that motor features formed during word learning could causally affect embodiment in the motor system for abstract verbs, and reactivation of motor experiences in abstract verb processing depends on a given task’s demands. Our study supports the view that conceptual representations, even abstract concepts, can be grounded in sensorimotor experiences.
Collapse
Affiliation(s)
- Xiang Li
- Academy of Psychology and Behavior, Tianjin Normal University, Tianjin, China
- Faculty of Psychology, Tianjin Normal University, Tianjin, China
- Department of Psychology, Xinxiang Medical University, Xinxiang, China
| | - Dan Luo
- Academy of Psychology and Behavior, Tianjin Normal University, Tianjin, China
- Faculty of Psychology, Tianjin Normal University, Tianjin, China
- Faculty of Education, Henan Normal University, Xinxiang, China
| | - Chao Wang
- Academy of Psychology and Behavior, Tianjin Normal University, Tianjin, China
- Faculty of Psychology, Tianjin Normal University, Tianjin, China
| | - Yaoyuan Xia
- Department of Physical Education, Zhejiang University of Finance and Economics, Hangzhou, China
| | - Hua Jin
- Academy of Psychology and Behavior, Tianjin Normal University, Tianjin, China
- Faculty of Psychology, Tianjin Normal University, Tianjin, China
- *Correspondence: Hua Jin,
| |
Collapse
|