A computational model of perception and action for cognitive robotics

How the conception of control influences our understanding of actions

Wilful movement requires neural control. Commonly, neural computations are thought to generate motor commands that bring the musculoskeletal system - that is, the plant - from its current physical state into a desired physical state. The current state can be estimated from past motor commands and from sensory information. Modelling movement on the basis of this concept of plant control strives to explain behaviour by identifying the computational principles for control signals that can reproduce the observed features of movements. From an alternative perspective, movements emerge in a dynamically coupled agent-environment system from the pursuit of subjective perceptual goals. Modelling movement on the basis of this concept of perceptual control aims to identify the controlled percepts and their coupling rules that can give rise to the observed characteristics of behaviour. In this Perspective, we discuss a broad spectrum of approaches to modelling human motor control and their notions of control signals, internal models, handling of sensory feedback delays and learning. We focus on the influence that the plant control and the perceptual control perspective may have on decisions when modelling empirical data, which may in turn shape our understanding of actions.

Toward an idiomatic framework for cognitive robotics

Inspired by the “cognitive hourglass” model presented by the researchers behind the cognitive architecture called Sigma, we propose a framework for developing cognitive architectures for cognitive robotics. The main purpose of the proposed framework is to ease development of cognitive architectures by encouraging cooperation and re-use of existing results. This is done by proposing a framework dividing development of cognitive architectures into a series of layers that can be considered partly in isolation, some of which directly relate to other research fields. Finally, we introduce and review some topics essential for the proposed framework. We also outline a set of applications.

•

The proposed framework divides development of cognitive architectures into layers

•

The framework spans the best-known approaches employed within cognitive robotics

•

The framework is centered around modern probabilistic programming techniques

•

Two applications demonstrate the concepts of the framework

For many decades, robots have been expected to transfigure the world we live in, and in many ways they already have, by increasingly taking over dull, dirty, and dangerous jobs. However, for robots to integrate fully and seamlessly into human societies, robots need to be able to learn and reason from experience and effectively deal with unpredictable and dynamic environments. Developing robotic systems with such intelligence is a tremendous and difficult task, which has led to the foundation of the new multi-disciplinary scientific field called cognitive robotics, merging research in adaptive robotics, cognitive science, and artificial intelligence. To ease merging research from these scientific fields, we propose a general framework for developing intelligent robotic systems based on recent advancements in the machine learning community. We hope that this framework will aid researchers and practitioners in bringing even more helpful robots into our societies.

Vector Symbolic Architectures for Context-Free Grammars

Vector symbolic architectures (VSA) are a viable approach for the hyperdimensional representation of symbolic data, such as documents, syntactic structures, or semantic frames. We present a rigorous mathematical framework for the representation of phrase structure trees and parse trees of context-free grammars (CFG) in Fock space, i.e. infinite-dimensional Hilbert space as being used in quantum field theory. We define a novel normal form for CFG by means of term algebras. Using a recently developed software toolbox, called FockBox, we construct Fock space representations for the trees built up by a CFG left-corner (LC) parser. We prove a universal representation theorem for CFG term algebras in Fock space and illustrate our findings through a low-dimensional principal component projection of the LC parser state. Our approach could leverage the development of VSA for explainable artificial intelligence (XAI) by means of hyperdimensional deep neural computation.

Building the Foundation of Robot Explanation Generation Using Behavior Trees

As autonomous robots continue to be deployed near people, robots need to be able to explain their actions. In this article, we focus on organizing and representing complex tasks in a way that makes them readily explainable. Many actions consist of sub-actions, each of which may have several sub-actions of their own, and the robot must be able to represent these complex actions before it can explain them. To generate explanations for robot behavior, we propose using Behavior Trees (BTs), which are a powerful and rich tool for robot task specification and execution. However, for BTs to be used for robot explanations, their free-form, static structure must be adapted. In this work, we add structure to previously free-form BTs by framing them as a set of semantic sets {goal, subgoals, steps, actions} and subsequently build explanation generation algorithms that answer questions seeking causal information about robot behavior. We make BTs less static with an algorithm that inserts a subgoal that satisfies all dependencies. We evaluate our BTs for robot explanation generation in two domains: a kitting task to assemble a gearbox, and a taxi simulation. Code for the behavior trees (in XML) and all the algorithms is available at github.com/uml-robotics/robot-explanation-BTs.

Modeling Cognitive Activity of the Human Brain by the Mathematical Apparatus of Quantum Mechanics

This paper discusses the possible approaches to modeling the cognitive activity of the human brain using the mathematical apparatus of quantum mechanics (primarily – potential wells and virtual particles) in terms of the theory of information representations. The article briefly describes the proposed theory of information representations, draws analogies, and identifies common features of information representations of the human mind and Feynman’s virtual particles. The human mind is represented as a one-dimensional potential well with finite walls of different sizes and internal potential barrier simulating the boundary between consciousness and subconsciousness. This creates a foundation for a mathematical apparatus that can make it possible to forecast particular cognitive functions of the human brain. The results of these studies can be used to create predictive models of various cognitive disorders (diseases) and to be used in diagnostics.

An Episodic Model of Task Switching Effects: Erasing the Homunculus from Memory

The Parallel Episodic Processing (PEP) model is a neural network for simulating human performance in speeded response time tasks. It learns with an exemplar-based memory store and it is capable of modelling findings from various subdomains of cognition. In this paper, we show how the PEP model can be designed to follow instructions (e.g., task rules and goals). The extended PEP model is then used to simulate a number of key findings from the task switching domain. These include the switch cost, task-rule congruency effects, response repetition asymmetries, cue repetition benefits, and the full pattern of means from a recent feature integration decomposition of cued task switching (Schmidt & Liefooghe, 2016). We demonstrate that the PEP model fits the participant data well, that the model does not possess the flexibility to match any pattern of results, and that a number of competing task switching models fail to account for key observations that the PEP model produces naturally. Given the parsimony and unique explanatory power of the episodic account presented here, our results suggest that feature-integration biases have a far greater power in explaining task-switching performance than previously assumed.

A Survey of Behavioral Models for Social Robots

The cooperation between humans and robots is becoming increasingly important in our society. Consequently, there is a growing interest in the development of models that can enhance and enrich the interaction between humans and robots. A key challenge in the Human-Robot Interaction (HRI) field is to provide robots with cognitive and affective capabilities, by developing architectures that let them establish empathetic relationships with users. Over the last several years, multiple models were proposed to face this open-challenge. This work provides a survey of the most relevant attempts/works. In details, it offers an overview of the architectures present in literature focusing on three specific aspects of HRI: the development of adaptive behavioral models, the design of cognitive architectures, and the ability to establish empathy with the user. The research was conducted within two databases: Scopus and Web of Science. Accurate exclusion criteria were applied to screen the 4916 articles found. At the end, 56 articles were selected. For each work, an evaluation of the model is made. Pros and cons of each work are detailed by analyzing the aspects that can be improved to establish an enjoyable interaction between robots and users.

Modeling of cognitive brain activity through the information images theory in terms of the bilingual Stroop test

The paper presents the basic principles of the information image theory and a mathematical model developed using it. The hierarchy of information images in an individual mind, which determines his∖her real and virtual activity, is considered. Algorithms for describing transfer and distortion of information images by individuals in the communication process are constructed.

To corroborate the theory experimentally, the bilingual Stroop test is used. The results of the test are interpreted using the introduced theory, and are then compared with the results of computer modeling based on the theory. It is shown that information images can be used not only to explain a number of cognitive processes of the human mind, but also to predict their dynamics in a number of particular cases.

The test of both worlds: identifying feature binding and control processes in congruency sequence tasks by means of action dynamics

Cognitive control processes enable us to act flexibly in a world posing ever-changing demands on our cognitive system. To study cognitive control, conflict tasks and especially congruency sequence effects have been regarded as a fruitful tool. However, for the last decade a dispute has arisen whether or not congruency sequence effects are indeed a valid measure of cognitive control processes. This debate has led to the development of increasingly complex paradigms involving numerous, intricately designed experimental conditions which are aimed at excluding low-level, associative learning mechanisms like feature binding as an alternative explanation for the emergence of congruency sequence effects. Here, we try to go beyond this all-or-nothing thinking by investigating the assumption that both cognitive control processes as well as feature binding mechanisms occur within trials of the same task. Based on a theoretical dual-route-model of behavior under conflict, we show that both classes of cognitive mechanisms should affect behavior at different points of the decision process. By comparing these predictions to continuous mouse movements from an adapted Simon task, we find evidence that control processes and feature binding mechanisms do indeed coexist within the task but that they follow distinct timing patterns. We argue that this dynamic approach to cognitive processing opens up new ways to investigate the diversity of co-existing processes that contribute to the selection of behavior.

HiTEC: a connectionist model of the interaction between perception and action planning

Increasing evidence suggests that perception and action planning do not represent separable stages of a unidirectional processing sequence, but rather emerging properties of highly interactive processes. To capture these characteristics of the human cognitive system, we have developed a connectionist model of the interaction between perception and action planning: HiTEC, based on the Theory of Event Coding (Hommel et al. in Behav Brain Sci 24:849–937, 2001). The model is characterized by representations at multiple levels and by shared representations and processes. It complements available models of stimulus–response translation by providing a rationale for (1) how situation-specific meanings of motor actions emerge, (2) how and why some aspects of stimulus–response translation occur automatically and (3) how task demands modulate sensorimotor processing. The model is demonstrated to provide a unitary account and simulation of a number of key findings with multiple experimental paradigms on the interaction between perception and action such as the Simon effect, its inversion (Hommel in Psychol Res 55:270–279, 1993), and action–effect learning.

The theory of information images: Modeling based on diffusion equations

This work represents a formalized description of information and communicative interactions of individuals on the basis of theory of information images (II). It also demonstrates how important it is to choose the models type adequate to the systems under research. It also introduces an explication of the possibility to create the model of information and communication interactions that can illustrate transmission of information between two and more individuals. Methods and approaches suggested in this paper allow us to compare different levels of the described processes depending on the chosen architecture of the model; the methods and approaches mentioned in the current work can be also used to simulate the processes of distortion and generation of information images while information and communication social interaction. Expansion and addition of information images theory in terms of transmission of information among individuals enables us to speak about the space of individual II. The existence of such a space and creation of correct formalized model help us to explain a number of characteristic phenomena of human thinking processes. As a result of this research, the authors introduce an equation that describes the spatial and structural evolution of individual II. There is also an example of modeling on the basis of this theory taking into account the results of the experiment (bilingual Stroop test).

A continuous-time neural model for sequential action

Action selection, planning and execution are continuous processes that evolve over time, responding to perceptual feedback as well as evolving top-down constraints. Existing models of routine sequential action (e.g. coffee- or pancake-making) generally fall into one of two classes: hierarchical models that include hand-built task representations, or heterarchical models that must learn to represent hierarchy via temporal context, but thus far lack goal-orientedness. We present a biologically motivated model of the latter class that, because it is situated in the Leabra neural architecture, affords an opportunity to include both unsupervised and goal-directed learning mechanisms. Moreover, we embed this neurocomputational model in the theoretical framework of the theory of event coding (TEC), which posits that actions and perceptions share a common representation with bidirectional associations between the two. Thus, in this view, not only does perception select actions (along with task context), but actions are also used to generate perceptions (i.e. intended effects). We propose a neural model that implements TEC to carry out sequential action control in hierarchically structured tasks such as coffee-making. Unlike traditional feedforward discrete-time neural network models, which use static percepts to generate static outputs, our biological model accepts continuous-time inputs and likewise generates non-stationary outputs, making short-timescale dynamic predictions.

How Task Goals Mediate the Interplay between Perception and Action

Theories of embodied cognition suppose that perception, action, and cognition are tightly intertwined and share common representations and processes. Indeed, numerous empirical studies demonstrate interaction between stimulus perception, response planning, and response execution. In this paper, we present an experiment and a connectionist model that show how the Simon effect, a canonical example of perception–action congruency, can be moderated by the (cognitive representation of the) task instruction. To date, no representational account of this influence exists. In the experiment, a two-dimensional Simon task was used, with critical stimuli being colored arrows pointing in one of four directions (backward, forward, left, or right). Participants stood on a Wii balance board, oriented diagonally toward the screen displaying the stimuli. They were either instructed to imagine standing on a snowboard or on a pair of skis and to respond to the stimulus color by leaning toward either the left or right foot. We expected that participants in the snowboard condition would encode these movements as forward or backward, resulting in a Simon effect on this dimension. This was confirmed by the results. The left–right congruency effect was larger in the ski condition, whereas the forward–backward congruency effect appeared only in the snowboard condition. The results can be readily accounted for by HiTEC, a connectionist model that aims at capturing the interaction between perception and action at the level of representations, and the way this interaction is mediated by cognitive control. Together, the empirical work and the connectionist model contribute to a better understanding of the complex interaction between perception, cognition, and action.