1
|
Sobhani M, Smith J, Pipe A, Peer A. A Novel Mirror Neuron Inspired Decision-Making Architecture for Human–Robot Interaction. Int J Soc Robot 2023. [DOI: 10.1007/s12369-023-00988-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/29/2023]
Abstract
AbstractInspired by the role of mirror neurons and the importance of predictions in joint action, a novel decision-making structure is proposed, designed and tested for both individual and dyadic action. The structure comprises models representing individual decision policies, policy integration layer(s), and a negotiation layer. The latter is introduced to prevent and resolve conflicts among individuals through internal simulation rather than via explicit agent-agent communication. As the main modelling tool, Dynamic Neural Fields (DNFs) were chosen. Data was captured from human-human experiments with a decision-making task performed by either one or two participants. The task involves choosing and picking blocks one by one from seven wooden blocks to create an alpha/numeric character on a 7-segment. The task is designed to be as generic as possible. Recorded hand and blocks movements were used for developing DNF-based models by optimising parameters using a genetic algorithm. Results show that decision policies can be modelled and integrated with acceptable accuracy for individual performances. In the dyadic experiment, using only individual models without the negotiation layer, the model failed to resolve conflicts. However, with the implementation of a negotiation layer, this problem could be overcome. The proposed decision-making structure based on DNFs is developed and tested for a simple pick-and-place task. However, the main primitive underlying action of this task, pick-and-place, is indeed part of many more complex tasks people perform in their day-to-day life. Paired with the possibility to gradually evolve the architecture by adding new policies on demand, the architecture provides a general framework for modelling decision-making in joint action tasks.
Collapse
|
2
|
Sadeghi S, Schmidt SNL, Mier D, Hass J. Effective Connectivity of the Human Mirror Neuron System During Social Cognition. Soc Cogn Affect Neurosci 2022; 17:732-743. [PMID: 35086135 PMCID: PMC9340111 DOI: 10.1093/scan/nsab138] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/09/2021] [Revised: 11/15/2021] [Accepted: 01/27/2022] [Indexed: 11/17/2022] Open
Abstract
The human mirror neuron system (MNS) can be considered the neural basis of social cognition. Identifying the global network structure of this system can provide significant progress in the field. In this study, we use dynamic causal modeling (DCM) to determine the effective connectivity between central regions of the MNS for the first time during different social cognition tasks. Sixty-seven healthy participants completed fMRI scanning while performing social cognition tasks, including imitation, empathy and theory of mind. Superior temporal sulcus (STS), inferior parietal lobule (IPL) and Brodmann area 44 (BA44) formed the regions of interest for DCM. Varying connectivity patterns, 540 models were built and fitted for each participant. By applying group-level analysis, Bayesian model selection and Bayesian model averaging, the optimal family and model for all experimental tasks were found. For all social-cognitive processes, effective connectivity from STS to IPL and from STS to BA44 was found. For imitation, additional mutual connections occurred between STS and BA44, as well as BA44 and IPL. The results suggest inverse models in which the motor regions BA44 and IPL receive sensory information from the STS. In contrast, for imitation, a sensory loop with an exchange of motor-to-sensory and sensory-to-motor information seems to exist.
Collapse
Affiliation(s)
- Sadjad Sadeghi
- Department of Theoretical Neuroscience, Central Institute of Mental Health, Mannheim 68159, Germany
- Department of Physics and Astronomy, Heidelberg University, Heidelberg 69120, Germany
| | | | | | - Joachim Hass
- Correspondence should be addressed to Joachim Hass, Faculty of Applied Psychology, SRH University of Applied Sciences, Maria-Probst-Strasse 3A, Heidelberg 69123, Germany. E-mail:
| |
Collapse
|
3
|
Yates L, Hobson H. Continuing to look in the mirror: A review of neuroscientific evidence for the broken mirror hypothesis, EP-M model and STORM model of autism spectrum conditions. AUTISM : THE INTERNATIONAL JOURNAL OF RESEARCH AND PRACTICE 2020; 24:1945-1959. [PMID: 32668956 PMCID: PMC7539595 DOI: 10.1177/1362361320936945] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/23/2022]
Abstract
The mirror neuron system has been argued to be a key brain system responsible for action understanding and imitation. Subsequently, mirror neuron system dysfunction has therefore been proposed to explain the social deficits manifested within autism spectrum condition, an approach referred to as the broken mirror hypothesis. Despite excitement surrounding this hypothesis, extensive research has produced insufficient evidence to support the broken mirror hypothesis in its pure form, and instead two alternative models have been formulated: EP-M model and the social top-down response modulation (STORM) model. All models suggest some dysfunction regarding the mirror neuron system in autism spectrum condition, be that within the mirror neuron system itself or systems that regulate the mirror neuron system. This literature review compares these three models in regard to recent neuroscientific investigations. This review concludes that there is insufficient support for the broken mirror hypothesis, but converging evidence supports an integrated EP-M and STORM model.
Collapse
|
4
|
Giese MA, Rizzolatti G. Neural and Computational Mechanisms of Action Processing: Interaction between Visual and Motor Representations. Neuron 2016; 88:167-80. [PMID: 26447579 DOI: 10.1016/j.neuron.2015.09.040] [Citation(s) in RCA: 32] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/23/2022]
Abstract
Action recognition has received enormous interest in the field of neuroscience over the last two decades. In spite of this interest, the knowledge in terms of fundamental neural mechanisms that provide constraints for underlying computations remains rather limited. This fact stands in contrast with a wide variety of speculative theories about how action recognition might work. This review focuses on new fundamental electrophysiological results in monkeys, which provide constraints for the detailed underlying computations. In addition, we review models for action recognition and processing that have concrete mathematical implementations, as opposed to conceptual models. We think that only such implemented models can be meaningfully linked quantitatively to physiological data and have a potential to narrow down the many possible computational explanations for action recognition. In addition, only concrete implementations allow judging whether postulated computational concepts have a feasible implementation in terms of realistic neural circuits.
Collapse
Affiliation(s)
- Martin A Giese
- Section on Computational Sensomotorics, Hertie Institute for Clinical Brain Research & Center for Integrative Neuroscience, University Clinic Tübingen, Otfried-Müller Str. 25, 72076 Tübingen, Germany.
| | - Giacomo Rizzolatti
- IIT Brain Center for Social and Motor Cognition, 43100, Parma, Italy; Dipartimento di Neuroscienze, Università di Parma, 43100 Parma, Italy.
| |
Collapse
|
5
|
Abstract
Complementary colors are color pairs which, when combined in the right proportions, produce white or black. Complementary actions refer here to forms of social interaction wherein individuals adapt their joint actions according to a common aim. Notably, complementary actions are incongruent actions. But being incongruent is not sufficient to be complementary (i.e., to complete the action of another person). Successful complementary interactions are founded on the abilities: (i) to simulate another person's movements, (ii) to predict another person's future action/s, (iii) to produce an appropriate incongruent response which differ, while interacting, with observed ones, and (iv) to complete the social interaction by integrating the predicted effects of one's own action with those of another person. This definition clearly alludes to the functional importance of complementary actions in the perception-action cycle and prompts us to scrutinize what is taking place behind the scenes. Preliminary data on this topic have been provided by recent cutting-edge studies utilizing different research methods. This mini-review aims to provide an up-to-date overview of the processes and the specific activations underlying complementary actions.
Collapse
Affiliation(s)
- Luisa Sartori
- Dipartimento di Psicologia Generale, Università di Padova , Padova, Italy ; Cognitive Neuroscience Center, Università di Padova , Padova, Italy
| | - Sonia Betti
- Dipartimento di Psicologia Generale, Università di Padova , Padova, Italy
| |
Collapse
|
6
|
Demiris Y, Aziz-Zadeh L, Bonaiuto J. Information processing in the mirror neuron system in primates and machines. Neuroinformatics 2014; 12:63-91. [PMID: 24085487 DOI: 10.1007/s12021-013-9200-7] [Citation(s) in RCA: 20] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
Abstract
The mirror neuron system in primates matches observations of actions with the motor representations used for their execution, and is a topic of intense research and debate in biological and computational disciplines. In robotics, models of this system have been used for enabling robots to imitate and learn how to perform tasks from human demonstrations. Yet, existing computational and robotic models of these systems are found in multiple levels of description, and although some models offer plausible explanations and testable predictions, the difference in the granularity of the experimental setups, methodologies, computational structures and selected modeled data make principled meta-analyses, common in other fields, difficult. In this paper, we adopt an interdisciplinary approach, using the BODB integrated environment in order to bring together several different but complementary computational models, by functionally decomposing them into brain operating principles (BOPs) which each capture a limited subset of the model's functionality. We then explore links from these BOPs to neuroimaging and neurophysiological data in order to pinpoint complementary and conflicting explanations and compare predictions against selected sets of neurobiological data. The results of this comparison are used to interpret mirror system neuroimaging results in terms of neural network activity, evaluate the biological plausibility of mirror system models, and suggest new experiments that can shed light on the neural basis of mirror systems.
Collapse
Affiliation(s)
- Yiannis Demiris
- Department of Electrical and Electronic Engineering, Imperial College London, London, UK,
| | | | | |
Collapse
|
7
|
A neuroinformatics of brain modeling and its implementation in the Brain Operation Database BODB. Neuroinformatics 2014; 12:5-26. [PMID: 24234915 DOI: 10.1007/s12021-013-9209-y] [Citation(s) in RCA: 8] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/26/2022]
Abstract
We present principles for an integrated neuroinformatics framework which makes explicit how models are grounded on empirical evidence, explain (or not) existing empirical results and make testable predictions. The new ontological framework makes explicit how models bring together structural, functional, and related empirical observations. We emphasize schematics of the model’s operation linked to summaries of empirical data (SEDs) used in both the design and testing of the model, with tests comparing SEDs to summaries of simulation results (SSRs) from the model. We stress the importance of protocols for models as well as experiments. We complement the structural ontology of nested brain structures with a functional ontology of Brain Operating Principles (BOPs) for observed neural function and an ontological framework for grounding models in empirical data. We present an implementation of this ontological framework in the Brain Operation Database (BODB), an environment in which modelers and experimentalists can work together by making use of their shared empirical data, models and expertise.
Collapse
|
8
|
Pezzulo G, Candidi M, Dindo H, Barca L. Action simulation in the human brain: Twelve questions. NEW IDEAS IN PSYCHOLOGY 2013. [DOI: 10.1016/j.newideapsych.2013.01.004] [Citation(s) in RCA: 45] [Impact Index Per Article: 4.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/13/2023]
|
9
|
Zhang N, Campo S, Janz KF, Eckler P, Yang J, Snetselaar LG, Signorini A. Electronic word of mouth on twitter about physical activity in the United States: exploratory infodemiology study. J Med Internet Res 2013; 15:e261. [PMID: 24257325 PMCID: PMC3841353 DOI: 10.2196/jmir.2870] [Citation(s) in RCA: 36] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/08/2013] [Revised: 09/24/2013] [Accepted: 11/06/2013] [Indexed: 11/16/2022] Open
Abstract
Background Twitter is a widely used social medium. However, its application in promoting health behaviors is understudied. Objective In order to provide insights into designing health marketing interventions to promote physical activity on Twitter, this exploratory infodemiology study applied both social cognitive theory and the path model of online word of mouth to examine the distribution of different electronic word of mouth (eWOM) characteristics among personal tweets about physical activity in the United States. Methods This study used 113 keywords to retrieve 1 million public tweets about physical activity in the United States posted between January 1 and March 31, 2011. A total of 30,000 tweets were randomly selected and sorted based on numbers generated by a random number generator. Two coders scanned the first 16,100 tweets and yielded 4672 (29.02%) tweets that they both agreed to be about physical activity and were from personal accounts. Finally, 1500 tweets were randomly selected from the 4672 tweets (32.11%) for further coding. After intercoder reliability scores reached satisfactory levels in the pilot coding (100 tweets separate from the final 1500 tweets), 2 coders coded 750 tweets each. Descriptive analyses, Mann-Whitney U tests, and Fisher exact tests were performed. Results Tweets about physical activity were dominated by neutral sentiments (1270/1500, 84.67%). Providing opinions or information regarding physical activity (1464/1500, 97.60%) and chatting about physical activity (1354/1500, 90.27%) were found to be popular on Twitter. Approximately 60% (905/1500, 60.33%) of the tweets demonstrated users’ past or current participation in physical activity or intentions to participate in physical activity. However, social support about physical activity was provided in less than 10% of the tweets (135/1500, 9.00%). Users with fewer people following their tweets (followers) (P=.02) and with fewer accounts that they followed (followings) (P=.04) were more likely to talk positively about physical activity on Twitter. People with more followers were more likely to post neutral tweets about physical activity (P=.04). People with more followings were more likely to forward tweets (P=.04). People with larger differences between number of followers and followings were more likely to mention companionship support for physical activity on Twitter (P=.04). Conclusions Future health marketing interventions promoting physical activity should segment Twitter users based on their number of followers, followings, and gaps between the number of followers and followings. The innovative application of both marketing and public health theory to examine tweets about physical activity could be extended to other infodemiology or infoveillance studies on other health behaviors (eg, vaccinations).
Collapse
Affiliation(s)
- Ni Zhang
- The University of Iowa alumnus, Iowa City, IA, United States.
| | | | | | | | | | | | | |
Collapse
|
10
|
Abstract
The visual recognition of actions is an important visual function that is critical for motor learning and social communication. Action-selective neurons have been found in different cortical regions, including the superior temporal sulcus, parietal and premotor cortex. Among those are mirror neurons, which link visual and motor representations of body movements. While numerous theoretical models for the mirror neuron system have been proposed, the computational basis of the visual processing of goal-directed actions remains largely unclear. While most existing models focus on the possible role of motor representations in action recognition, we propose a model showing that many critical properties of action-selective visual neurons can be accounted for by well-established visual mechanisms. Our model accomplishes the recognition of hand actions from real video stimuli, exploiting exclusively mechanisms that can be implemented in a biologically plausible way by cortical neurons. We show that the model provides a unifying quantitatively consistent account of a variety of electrophysiological results from action-selective visual neurons. In addition, it makes a number of predictions, some of which could be confirmed in recent electrophysiological experiments.
Collapse
|
11
|
Sciutti A, Bisio A, Nori F, Metta G, Fadiga L, Pozzo T, Sandini G. Measuring Human-Robot Interaction Through Motor Resonance. Int J Soc Robot 2012. [DOI: 10.1007/s12369-012-0143-1] [Citation(s) in RCA: 56] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
|
12
|
Thill S, Svensson H, Ziemke T. Modeling the Development of Goal-Specificity in Mirror Neurons. Cognit Comput 2011; 3:525-538. [PMID: 22207881 PMCID: PMC3228964 DOI: 10.1007/s12559-011-9108-1] [Citation(s) in RCA: 9] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/22/2011] [Accepted: 08/26/2011] [Indexed: 11/29/2022]
Abstract
Neurophysiological studies have shown that parietal mirror neurons encode not only actions but also the goal of these actions. Although some mirror neurons will fire whenever a certain action is perceived (goal-independently), most will only fire if the motion is perceived as part of an action with a specific goal. This result is important for the action-understanding hypothesis as it provides a potential neurological basis for such a cognitive ability. It is also relevant for the design of artificial cognitive systems, in particular robotic systems that rely on computational models of the mirror system in their interaction with other agents. Yet, to date, no computational model has explicitly addressed the mechanisms that give rise to both goal-specific and goal-independent parietal mirror neurons. In the present paper, we present a computational model based on a self-organizing map, which receives artificial inputs representing information about both the observed or executed actions and the context in which they were executed. We show that the map develops a biologically plausible organization in which goal-specific mirror neurons emerge. We further show that the fundamental cause for both the appearance and the number of goal-specific neurons can be found in geometric relationships between the different inputs to the map. The results are important to the action-understanding hypothesis as they provide a mechanism for the emergence of goal-specific parietal mirror neurons and lead to a number of predictions: (1) Learning of new goals may mostly reassign existing goal-specific neurons rather than recruit new ones; (2) input differences between executed and observed actions can explain observed corresponding differences in the number of goal-specific neurons; and (3) the percentage of goal-specific neurons may differ between motion primitives.
Collapse
Affiliation(s)
- Serge Thill
- Cognition and Interaction Lab, School of Humanities and Informatics, University of Skövde, P.O. Box 408, 541 28 Skövde, Sweden
| | | | | |
Collapse
|
13
|
van Rooij I, Kwisthout J, Blokpoel M, Szymanik J, Wareham T, Toni I. Intentional communication: computationally easy or difficult? Front Hum Neurosci 2011; 5:52. [PMID: 21747765 PMCID: PMC3129534 DOI: 10.3389/fnhum.2011.00052] [Citation(s) in RCA: 38] [Impact Index Per Article: 2.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/08/2010] [Accepted: 05/16/2011] [Indexed: 11/29/2022] Open
Abstract
Human intentional communication is marked by its flexibility and context sensitivity. Hypothesized brain mechanisms can provide convincing and complete explanations of the human capacity for intentional communication only insofar as they can match the computational power required for displaying that capacity. It is thus of importance for cognitive neuroscience to know how computationally complex intentional communication actually is. Though the subject of considerable debate, the computational complexity of communication remains so far unknown. In this paper we defend the position that the computational complexity of communication is not a constant, as some views of communication seem to hold, but rather a function of situational factors. We present a methodology for studying and characterizing the computational complexity of communication under different situational constraints. We illustrate our methodology for a model of the problems solved by receivers and senders during a communicative exchange. This approach opens the way to a principled identification of putative model parameters that control cognitive processes supporting intentional communication.
Collapse
Affiliation(s)
- Iris van Rooij
- Donders Institute for Brain, Cognition and Behaviour, Radboud University Nijmegen Nijmegen, Netherlands
| | | | | | | | | | | |
Collapse
|
14
|
|
15
|
Spatiotemporal tuning of the facilitation of biological motion perception by concurrent motor execution. J Neurosci 2011; 31:3493-9. [PMID: 21368061 DOI: 10.1523/jneurosci.4277-10.2011] [Citation(s) in RCA: 38] [Impact Index Per Article: 2.9] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/21/2022] Open
Abstract
The execution of motor behavior influences concurrent visual action observation and especially the perception of biological motion. The neural mechanisms underlying this interaction between perception and motor execution are not exactly known. In addition, the available experimental evidence is partially inconsistent because previous studies have reported facilitation as well as impairments of action perception by concurrent execution. Exploiting a novel virtual reality paradigm, we investigated the spatiotemporal tuning of the influence of motor execution on the perception of biological motion within a signal-detection task. Human observers were presented with point-light stimuli that were controlled by their own movements. Participants had to detect a point-light arm in a scrambled mask, either while executing waving movements or without concurrent motor execution (baseline). The temporal and spatial coherence between the observed and executed movements was parametrically varied. We found a systematic tuning of the facilitatory versus inhibitory influences of motor execution on biological motion detection with respect to the temporal and the spatial congruency between observed and executed movements. Specifically, we found a gradual transition between facilitatory and inhibitory interactions for decreasing temporal synchrony and spatial congruency. This result provides evidence for a spatiotemporally highly selective coupling between dynamic motor representations and neural structures involved in the visual processing of biological motion. In addition, our study offers a unifying explanation that reconciles contradicting results about modulatory effects of motor execution on biological motion perception in previous studies.
Collapse
|
16
|
Nadel J, Aouka N, Coulon N, Gras-Vincendon A, Canet P, Fagard J, Bursztejn C. Yes they can! An approach to observational learning in low-functioning children with autism. AUTISM : THE INTERNATIONAL JOURNAL OF RESEARCH AND PRACTICE 2011; 15:421-35. [PMID: 21454387 DOI: 10.1177/1362361310386508] [Citation(s) in RCA: 25] [Impact Index Per Article: 1.9] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
Learning by doing and learning by observing are two facets of the tight coupling between perception and action discovered at the brain level. Developmental studies of observational learning still remain rare and even more rare are studies documenting the capacities of low-functioning children with autism to learn by observation. In the first investigation of this question, twenty nonverbal children with autism with a developmental age of 24 and 36 months, and twenty matched typical children, were presented with an experimental box requiring that a hierarchical sequence of subgoals be performed before it could be opened. A 9-day testing procedure included four presentations of the red box and two video demonstrations of how to open it. Two scores were computed, one concerning the number of sub-goals fulfilled and the other the relevant manipulations of the material. Within-group analyses revealed that only the typical children learned partly or fully the sequence of subgoals after the first video-demonstration. The addition of a second demonstration allowed the two subgroups with autism to learn partly or fully the sequence of subgoals. The differences between learning to manipulate and learning to produce a goal are discussed in terms of relationships between understanding actions and understanding action-effect relations.
Collapse
Affiliation(s)
- Jacqueline Nadel
- Centre Emotion, CNRS USR3246, Hôpital de La Salpêtrière, Paris, France.
| | | | | | | | | | | | | |
Collapse
|
17
|
Bicho E, Erlhagen W, Louro L, e Silva EC. Neuro-cognitive mechanisms of decision making in joint action: a human-robot interaction study. Hum Mov Sci 2011; 30:846-68. [PMID: 21208673 DOI: 10.1016/j.humov.2010.08.012] [Citation(s) in RCA: 44] [Impact Index Per Article: 3.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/17/2009] [Revised: 08/09/2010] [Accepted: 08/13/2010] [Indexed: 11/26/2022]
Abstract
In this paper we present a model for action preparation and decision making in cooperative tasks that is inspired by recent experimental findings about the neuro-cognitive mechanisms supporting joint action in humans. It implements the coordination of actions and goals among the partners as a dynamic process that integrates contextual cues, shared task knowledge and predicted outcome of others' motor behavior. The control architecture is formalized by a system of coupled dynamic neural fields representing a distributed network of local but connected neural populations. Different pools of neurons encode task-relevant information about action means, task goals and context in the form of self-sustained activation patterns. These patterns are triggered by input from connected populations and evolve continuously in time under the influence of recurrent interactions. The dynamic model of joint action is evaluated in a task in which a robot and a human jointly construct a toy object. We show that the highly context sensitive mapping from action observation onto appropriate complementary actions allows coping with dynamically changing joint action situations.
Collapse
Affiliation(s)
- Estela Bicho
- Department of Industrial Electronics, University of Minho, 4800-058 Guimarães, Portugal.
| | | | | | | |
Collapse
|
18
|
Bicho E, Louro L, Erlhagen W. Integrating verbal and nonverbal communication in a dynamic neural field architecture for human-robot interaction. Front Neurorobot 2010; 4. [PMID: 20725504 PMCID: PMC2901089 DOI: 10.3389/fnbot.2010.00005] [Citation(s) in RCA: 22] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/01/2009] [Accepted: 04/27/2010] [Indexed: 11/13/2022] Open
Abstract
How do humans coordinate their intentions, goals and motor behaviors when performing joint action tasks? Recent experimental evidence suggests that resonance processes in the observer's motor system are crucially involved in our ability to understand actions of others’, to infer their goals and even to comprehend their action-related language. In this paper, we present a control architecture for human–robot collaboration that exploits this close perception-action linkage as a means to achieve more natural and efficient communication grounded in sensorimotor experiences. The architecture is formalized by a coupled system of dynamic neural fields representing a distributed network of neural populations that encode in their activation patterns goals, actions and shared task knowledge. We validate the verbal and nonverbal communication skills of the robot in a joint assembly task in which the human–robot team has to construct toy objects from their components. The experiments focus on the robot's capacity to anticipate the user's needs and to detect and communicate unexpected events that may occur during joint task execution.
Collapse
Affiliation(s)
- Estela Bicho
- Department of Industrial Electronics, University of Minho Guimarães, Portugal
| | | | | |
Collapse
|
19
|
Labiadh L, Ramanantsoa MM, Golomer E. Preschool-aged children’s jumps: Imitation performances. J Electromyogr Kinesiol 2010; 20:322-9. [DOI: 10.1016/j.jelekin.2009.05.005] [Citation(s) in RCA: 7] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/09/2007] [Revised: 04/29/2009] [Accepted: 05/20/2009] [Indexed: 11/27/2022] Open
|
20
|
The role of motor simulation in action perception: a neuropsychological case study. PSYCHOLOGICAL RESEARCH 2009; 73:477-85. [PMID: 19350271 PMCID: PMC2694935 DOI: 10.1007/s00426-009-0231-5] [Citation(s) in RCA: 21] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/17/2008] [Accepted: 02/23/2009] [Indexed: 11/01/2022]
Abstract
Research on embodied cognition stresses that bodily and motor processes constrain how we perceive others. Regarding action perception the most prominent hypothesis is that observed actions are matched to the observer's own motor representations. Previous findings demonstrate that the motor laws that constrain one's performance also constrain one's perception of others' actions. The present neuropsychological case study asked whether neurological impairments affect a person's performance and action perception in the same way. The results showed that patient DS, who suffers from a frontal brain lesion, not only ignored target size when performing movements but also when asked to judge whether others can perform the same movements. In other words DS showed the same violation of Fitts's law when performing and observing actions. These results further support the assumption of close perception action links and the assumption that these links recruit predictive mechanisms residing in the motor system.
Collapse
|
21
|
|
22
|
Hayes SJ, Ashford D, Bennett SJ. Goal-directed imitation: the means to an end. Acta Psychol (Amst) 2008; 127:407-15. [PMID: 17880901 DOI: 10.1016/j.actpsy.2007.07.009] [Citation(s) in RCA: 25] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/18/2006] [Revised: 07/20/2007] [Accepted: 07/23/2007] [Indexed: 11/30/2022] Open
Abstract
The effects of goal-directed imitation and observational learning were examined whilst learning a goal-directed motor skill (three-ball cascade juggling). An observational learning (OL) group observed a model and a control (CON) group received minimal verbal instructions regarding how to hold and release the juggling balls. The OL group performed more juggling cycles across practice and retention than the CON group. In addition, the OL group's upper limb coordination and ball flight trajectory pattern were more similar to the model's movements than the CON group. These data show that when the to-be-learnt movement pattern and end-goal are not specified by the task's mechanical constraints, or can be achieved by modifying a pre-existing motor skill, individuals have difficulty learning on the basis of discovery processes alone. Under these circumstances, observational learning is effective because it conveys to the individual the specific means by which the end-goal can be achieved. These findings lead us to suggest that when the end-goal and the means to achieve the end-goal are directly linked, the means are given sufficient weight in the goal hierarchy such that the model's movement is imitated.
Collapse
Affiliation(s)
- Spencer J Hayes
- Research Institute for Sport and Exercise Sciences, Liverpool John Moores University, Henry Cotton Campus, 15-21 Webster Street, Liverpool L3 2ET, UK.
| | | | | |
Collapse
|
23
|
Abstract
In this article, we examine the question of what information is processed during observational learning by evaluating a variety of methods, theories, and empirical data. Initially, we review work involving neuroimaging techniques and infant imitation. We then evaluate data from behavioural experiments involving adults, wherein a variety of attempts have been made to isolate the critical or minimal information constraining the acquisition of coordination. This body of research has included comparisons between video and point-light displays, manipulations to the amount and type of information presented in the display, the collection of point-of-gaze data, and manipulations to the task context in terms of outcome goals. We conclude that observational learning is governed by specific features of the model's action (i.e. motions of the end effector) and the task (i.e. outcome constraints) and, in contrast with traditional theoretical modelling, more global aspects of a model (i.e. the relative motions within and between joints) do not appear to be the primary method for constraining action execution.
Collapse
Affiliation(s)
- Nicola J Hodges
- School of Human Kinetics, University of British Columbia, Vancouver, Canada.
| | | | | | | |
Collapse
|
24
|
Abstract
This tutorial presents an architecture for autonomous robots to generate behavior in joint action tasks. To efficiently interact with another agent in solving a mutual task, a robot should be endowed with cognitive skills such as memory, decision making, action understanding and prediction. The proposed architecture is strongly inspired by our current understanding of the processing principles and the neuronal circuitry underlying these functionalities in the primate brain. As a mathematical framework, we use a coupled system of dynamic neural fields, each representing the basic functionality of neuronal populations in different brain areas. It implements goal-directed behavior in joint action as a continuous process that builds on the interpretation of observed movements in terms of the partner's action goal. We validate the architecture in two experimental paradigms: (1) a joint search task; (2) a reproduction of an observed or inferred end state of a grasping-placing sequence. We also review some of the mathematical results about dynamic neural fields that are important for the implementation work.
Collapse
Affiliation(s)
- Wolfram Erlhagen
- Department of Mathematics for Science and Technology, Universidade do Minho, 4800-058 Guimarães, Portugal.
| | | |
Collapse
|