Passive motion paradigm: an alternative to optimal control

Utilising redundancy in musculoskeletal systems for adaptive stiffness and muscle failure compensation: a model-free inverse statics approach

Vertebrates possess a biomechanical structure with redundant muscles, enabling adaptability in uncertain and complex environments. Harnessing this inspiration, musculoskeletal systems offer advantages like variable stiffness and resilience to actuator failure and fatigue. Despite their potential, the complex structure presents modelling challenges that are difficult to explicitly formulate and control. This difficulty arises from the need for comprehensive knowledge of the musculoskeletal system, including details such as muscle arrangement, and fully accessible muscle and joint states. Whilst existing model-free methods do not need explicit formulations, they also underutilise the benefits of muscle redundancy. Consequently, they necessitate retraining in the event of muscle failure and require manual tuning of parameters to control joint stiffness limiting their applications under unknown payloads. Presented here is a model-free local inverse statics controller for musculoskeletal systems, employing a feedforward neural network trained on motor babbling data. Experiments with a musculoskeletal leg model showcase the controller's adaptability to complex structures, including mono and bi-articulate muscles. The controller can compensate for changes such as weight variations, muscle failures, and environmental interactions, retaining reasonable accuracy without the need for any additional retraining.

Artificial cognition vs. artificial intelligence for next-generation autonomous robotic agents

The trend in industrial/service robotics is to develop robots that can cooperate with people, interacting with them in an autonomous, safe and purposive way. These are the fundamental elements characterizing the fourth and the fifth industrial revolutions (4IR, 5IR): the crucial innovation is the adoption of intelligent technologies that can allow the development of cyber-physical systems, similar if not superior to humans. The common wisdom is that intelligence might be provided by AI (Artificial Intelligence), a claim that is supported more by media coverage and commercial interests than by solid scientific evidence. AI is currently conceived in a quite broad sense, encompassing LLMs and a lot of other things, without any unifying principle, but self-motivating for the success in various areas. The current view of AI robotics mostly follows a purely disembodied approach that is consistent with the old-fashioned, Cartesian mind-body dualism, reflected in the software-hardware distinction inherent to the von Neumann computing architecture. The working hypothesis of this position paper is that the road to the next generation of autonomous robotic agents with cognitive capabilities requires a fully brain-inspired, embodied cognitive approach that avoids the trap of mind-body dualism and aims at the full integration of Bodyware and Cogniware. We name this approach Artificial Cognition (ACo) and ground it in Cognitive Neuroscience. It is specifically focused on proactive knowledge acquisition based on bidirectional human-robot interaction: the practical advantage is to enhance generalization and explainability. Moreover, we believe that a brain-inspired network of interactions is necessary for allowing humans to cooperate with artificial cognitive agents, building a growing level of personal trust and reciprocal accountability: this is clearly missing, although actively sought, in current AI. The ACo approach is a work in progress that can take advantage of a number of research threads, some of them antecedent the early attempts to define AI concepts and methods. In the rest of the paper we will consider some of the building blocks that need to be re-visited in a unitary framework: the principles of developmental robotics, the methods of action representation with prospection capabilities, and the crucial role of social interaction.

Embodied mechanisms of motor control in the octopus

Achieving complex behavior in soft-bodied animals is a hard task, because their body morphology is not constrained by a fixed number of jointed elements, as in skeletal animals, and thus the control system has to deal with practically an infinite number of control variables (degrees of freedom). Almost 30 years of research on Octopus vulgaris motor control has revealed that octopuses efficiently control their body with strategies that emerged during the adaptive coevolution of their nervous system and body morphology. In this minireview, we highlight principles of embodied organization that were revealed by studying octopus motor control, and that are used as inspiration for soft robotics. We describe the evolved solutions to the problem, implemented from the lowest level, the muscular system, to the network organization in higher motor control centers of the octopus brain. We show how the higher motor control centers, where the sensory-motor interface lies, can control and coordinate limbs with large degrees of freedom without using body-part maps to represent sensory and motor information, as they do in vertebrates. We demonstrate how this unique control mechanism, which allows efficient control of the body in a large variety of behaviors, is embodied within the animal's body morphology.

Biologically inspired robotic perception-action for soft fruit harvesting in vertical growing environments

Multiple interlinked factors like demographics, migration patterns, and economics are presently leading to the critical shortage of labour available for low-skilled, physically demanding tasks like soft fruit harvesting. This paper presents a biomimetic robotic solution covering the full 'Perception-Action' loop targeting harvesting of strawberries in a state-of-the-art vertical growing environment. The novelty emerges from both dealing with crop/environment variance as well as configuring the robot action system to deal with a range of runtime task constraints. Unlike the commonly used deep neural networks, the proposed perception system uses conditional Generative Adversarial Networks to identify the ripe fruit using synthetic data. The network can effectively train the synthetic data using the image-to-image translation concept, thereby avoiding the tedious work of collecting and labelling the real dataset. Once the harvest-ready fruit is localised using point cloud data generated by a stereo camera, our platform's action system can coordinate the arm to reach/cut the stem using the Passive Motion Paradigm framework inspired by studies on neural control of movement in the brain. Results from field trials for strawberry detection, reaching/cutting the stem of the fruit, and extension to analysing complex canopy structures/bimanual coordination (searching/picking) are presented. While this article focuses on strawberry harvesting, ongoing research towards adaptation of the architecture to other crops such as tomatoes and sweet peppers is briefly described.

Supplementary Information

The online version contains supplementary material available at 10.1007/s11119-023-10000-4.

A Vexing Question in Motor Control: The Degrees of Freedom Problem

The human “marionette” is extremely complex and multi-articulated: anatomical redundancy (in terms of Degrees of Freedom: DoFs), kinematic redundancy (movements can have different trajectories, velocities, and accelerations and yet achieve the same goal, according to the principle of Motor Equivalence), and neurophysiological redundancy (many more muscles than DoFs and multiple motor units for each muscle). Although it is quite obvious that such abundance is not noxious at all because, in contrast, it is instrumental for motor learning, allowing the nervous system to “explore” the space of feasible actions before settling on an elegant and possibly optimal solution, the crucial question then boils down to figure out how the nervous system “chooses/selects/recruits/modulates” task-dependent subsets of countless assemblies of DoFs as functional motor synergies. Despite this daunting conceptual riddle, human purposive behavior in daily life activities is a proof of concept that solutions can be found easily and quickly by the embodied brain of the human cognitive agent. The point of view suggested in this essay is to frame the question above in the old-fashioned but still seminal observation by Marr and Poggio that cognitive agents should be regarded as Generalized Information Processing Systems (GIPS) and should be investigated according to three nearly independent but complementary levels of analysis: 1) the computational level, 2) the algorithmic level, and 3) the implementation level. In this framework, we attempt to discriminate as well as aggregate the different hypotheses and solutions proposed so far: the optimal control hypothesis, the muscle synergy hypothesis, the equilibrium point hypothesis, or the uncontrolled manifold hypothesis, to mention the most popular ones. The proposed GIPS follows the strategy of factoring out shaping and timing by adopting a force-field based approach (the Passive Motion Paradigm) that is inspired by the Equilibrium Point Hypothesis, extended in such a way to represent covert as well overt actions. In particular, it is shown how this approach can explain spatio-temporal invariances and, at the same time, solve the Degrees of Freedom Problem.

Exploiting Spherical Projections To Generate Human-Like Wrist Pointing Movements

The mechanism behind the generation of human movements is of great interest in many fields (e.g. robotics and neuroscience) to improve therapies and technologies. Optimal Feedback Control (OFC) and Passive Motion Paradigm (PMP) are currently two leading theories capable of effectively producing human-like motions, but they require solving nonlinear inverse problems to find a solution. The main benefit of using PMP is the possibility of generating path-independent movements consistent with the stereotypical behaviour observed in humans, while the equivalent OFC formulation is path-dependent. Our results demonstrate how the path-independent behaviour observed for the wrist pointing task can be explained by spherical projections of the planar tasks. The combination of the projections with the fractal impedance controller eliminates the nonlinear inverse problem, which reduces the computational cost compared to previous methodologies. The motion exploits a recently proposed PMP architecture that replaces the nonlinear inverse optimisation with a nonlinear anisotropic stiffness impedance profile generated by the Fractal Impedance Controller, reducing the computational cost and not requiring a task-dependent optimisation.

Active listening

This paper introduces active listening, as a unified framework for synthesising and recognising speech. The notion of active listening inherits from active inference, which considers perception and action under one universal imperative: to maximise the evidence for our (generative) models of the world. First, we describe a generative model of spoken words that simulates (i) how discrete lexical, prosodic, and speaker attributes give rise to continuous acoustic signals; and conversely (ii) how continuous acoustic signals are recognised as words. The 'active' aspect involves (covertly) segmenting spoken sentences and borrows ideas from active vision. It casts speech segmentation as the selection of internal actions, corresponding to the placement of word boundaries. Practically, word boundaries are selected that maximise the evidence for an internal model of how individual words are generated. We establish face validity by simulating speech recognition and showing how the inferred content of a sentence depends on prior beliefs and background noise. Finally, we consider predictive validity by associating neuronal or physiological responses, such as the mismatch negativity and P300, with belief updating under active listening, which is greatest in the absence of accurate prior beliefs about what will be heard next.

The Effect of Crank Resistance on Arm Configuration and Muscle Activation Variances in Arm Cycling Movements

Arm cycling on an ergometer is common in sports training and rehabilitation protocols. The hand movement is constrained along a circular path, and the user is working against a resistance, maintaining a cadence. Even if the desired hand trajectory is given, there is the flexibility to choose patterns of joint coordination and muscle activation, given the kinematic redundancy of the upper limb. With changing external load, motor noise and changing joint stiffness may affect the pose of the arm even though the endpoint trajectory is unchanged. The objective of this study was to examine how the crank resistance influences the variances of joint configuration and muscle activation. Fifteen healthy participants performed arm cranking on an arm-cycle ergometer both unimanually and bimanually with a cadence of 60 rpm against three crank resistances. Joint configuration was represented in a 3-dimensional joint space defined by inter-segmental joint angles, while muscle activation in a 4-dimensional "muscle activation space" defined by EMGs of 4 arm muscles. Joint configuration variance in the course of arm cranking was not affected by crank resistance, whereas muscle activation variance was proportional to the square of muscle activation. The shape of the variance time profiles for both joint configuration and muscle activation was not affected by crank resistance. Contrary to the prevailing assumption that an increased motor noise would affect the variance of auxiliary movements, the influence of noise doesn't appear at the joint configuration level even when the system is redundant. Our results suggest the separation of kinematic- and force-control, via mechanisms that are compensating for dynamic nonlinearities. Arm cranking may be suitable when the aim is to perform training under different load conditions, preserving stable and secure control of joint movements and muscle activations.

Minds and Brains, Sleep and Psychiatry

Objective

This article offers a philosophical thesis for psychiatric disorders that rests upon some simple truths about the mind and brain. Specifically, it asks whether the dual aspect monism—that emerges from sleep research and theoretical neurobiology—can be applied to pathophysiology and psychopathology in psychiatry.

Methods

Our starting point is that the mind and brain are emergent aspects of the same (neuronal) dynamics; namely, the brain–mind. Our endpoint is that synaptic dysconnection syndromes inherit the same dual aspect; namely, aberrant inference or belief updating on the one hand, and a failure of neuromodulatory synaptic gain control on the other. We start with some basic considerations from sleep research that integrate the phenomenology of dreaming with the neurophysiology of sleep.

Results

We then leverage this treatment by treating the brain as an organ of inference. Our particular focus is on the role of precision (i.e., the representation of uncertainty) in belief updating and the accompanying synaptic mechanisms.

Conclusions

Finally, we suggest a dual aspect approach—based upon belief updating (i.e., mind processes) and its neurophysiological implementation (i.e., brain processes)—has a wide explanatory compass for psychiatry and various movement disorders. This approach identifies the kind of pathophysiology that underwrites psychopathology—and points to certain psychotherapeutic and psychopharmacological targets, which may stand in mechanistic relation to each other.

The ‘mind’ emerges from Bayesian belief updating in the ‘brain’

Psychopathology can be read as aberrant belief updating.

Aberrant belief updating follows from any neuromodulatory synaptopathy

Degeneracy and Redundancy in Active Inference

The notions of degeneracy and redundancy are important constructs in many areas, ranging from genomics through to network science. Degeneracy finds a powerful role in neuroscience, explaining key aspects of distributed processing and structure-function relationships in the brain. For example, degeneracy accounts for the superadditive effect of lesions on functional deficits in terms of a "many-to-one" structure-function mapping. In this paper, we offer a principled account of degeneracy and redundancy, when function is operationalized in terms of active inference, namely, a formulation of perception and action as belief updating under generative models of the world. In brief, "degeneracy" is quantified by the "entropy" of posterior beliefs about the causes of sensations, while "redundancy" is the "complexity" cost incurred by forming those beliefs. From this perspective, degeneracy and redundancy are complementary: Active inference tries to minimize redundancy while maintaining degeneracy. This formulation is substantiated using statistical and mathematical notions of degenerate mappings and statistical efficiency. We then illustrate changes in degeneracy and redundancy during the learning of a word repetition task. Finally, we characterize the effects of lesions-to intrinsic and extrinsic connections-using in silico disconnections. These numerical analyses highlight the fundamental difference between degeneracy and redundancy-and how they score distinct imperatives for perceptual inference and structure learning that are relevant to synthetic and biological intelligence.

Control of redundant pointing movements involving the wrist and forearm

The musculoskeletal system can move in more ways than are strictly necessary, allowing many tasks to be accomplished with a variety of limb configurations. Why some configurations are preferred has been a focus of motor control research, but most studies have focused on shoulder-elbow or whole arm movements. This study focuses on movements involving forearm pronation-supination (PS), wrist flexion-extension (FE), and wrist radial-ulnar deviation (RUD) and elucidates how these three degrees of freedom (DOF) combine to perform the common task of pointing, which only requires two DOF. Although pointing is more sensitive to FE and RUD than to PS and could be easily accomplished with FE and RUD alone, subjects tend to involve a small amount of PS. However, why we choose this behavior has been unknown and is the focus of this paper. With the use of a second-order model with lumped parameters, we tested a number of plausible control strategies involving minimization of work, potential energy, torque, and path length. None of these control schemes robustly predicted the observed behavior. However, an alternative control scheme, hypothesized to control the DOF that were most important to the task (FE and RUD) and ignore the less important DOF (PS), matched the observed behavior well. In particular, the behavior observed in PS appears to be a mechanical side effect caused by unopposed interaction torques. We conclude that moderately sized pointing movements involving the wrist and forearm are controlled by ignoring forearm rotation even though this strategy does not robustly minimize work, potential energy, torque, or path length. NEW & NOTEWORTHY Many activities require us to point our hands in a given direction using wrist and forearm rotations. Although there are infinitely many ways to do this, we tend to follow a stereotyped pattern. Why we choose this pattern has been unknown and is the focus of this paper. After testing a variety of hypotheses, we conclude that the pattern results from a simplifying strategy in which we focus on wrist rotations and ignore forearm rotation.

Goal-Directed Reasoning and Cooperation in Robots in Shared Workspaces: an Internal Simulation Based Neural Framework

From social dining in households to product assembly in manufacturing lines, goal-directed reasoning and cooperation with other agents in shared workspaces is a ubiquitous aspect of our day-to-day activities. Critical for such behaviours is the ability to spontaneously anticipate what is doable by oneself as well as the interacting partner based on the evolving environmental context and thereby exploit such information to engage in goal-oriented action sequences. In the setting of an industrial task where two robots are jointly assembling objects in a shared workspace, we describe a bioinspired neural architecture for goal-directed action planning based on coupled interactions between multiple internal models, primarily of the robot's body and its peripersonal space. The internal models (of each robot's body and peripersonal space) are learnt jointly through a process of sensorimotor exploration and then employed in a range of anticipations related to the feasibility and consequence of potential actions of two industrial robots in the context of a joint goal. The ensuing behaviours are demonstrated in a real-world industrial scenario where two robots are assembling industrial fuse-boxes from multiple constituent objects (fuses, fuse-stands) scattered randomly in their workspace. In a spatially unstructured and temporally evolving assembly scenario, the robots employ reward-based dynamics to plan and anticipate which objects to act on at what time instances so as to successfully complete as many assemblies as possible. The existing spatial setting fundamentally necessitates planning collision-free trajectories and avoiding potential collisions between the robots. Furthermore, an interesting scenario where the assembly goal is not realizable by either of the robots individually but only realizable if they meaningfully cooperate is used to demonstrate the interplay between perception, simulation of multiple internal models and the resulting complementary goal-directed actions of both robots. Finally, the proposed neural framework is benchmarked against a typically engineered solution to evaluate its performance in the assembly task. The framework provides a computational outlook to the emerging results from neurosciences related to the learning and use of body schema and peripersonal space for embodied simulation of action and prediction. While experiments reported here engage the architecture in a complex planning task specifically, the internal model based framework is domain-agnostic facilitating portability to several other tasks and platforms.

Social Cognition for Human-Robot Symbiosis-Challenges and Building Blocks

The next generation of robot companions or robot working partners will need to satisfy social requirements somehow similar to the famous laws of robotics envisaged by Isaac Asimov time ago (Asimov, 1942). The necessary technology has almost reached the required level, including sensors and actuators, but the cognitive organization is still in its infancy and is only partially supported by the current understanding of brain cognitive processes. The brain of symbiotic robots will certainly not be a “positronic” replica of the human brain: probably, the greatest part of it will be a set of interacting computational processes running in the cloud. In this article, we review the challenges that must be met in the design of a set of interacting computational processes as building blocks of a cognitive architecture that may give symbiotic capabilities to collaborative robots of the next decades: (1) an animated body-schema; (2) an imitation machinery; (3) a motor intentions machinery; (4) a set of physical interaction mechanisms; and (5) a shared memory system for incremental symbiotic development. We would like to stress that our approach is totally un-hierarchical: the five building blocks of the shared cognitive architecture are fully bi-directionally connected. For example, imitation and intentional processes require the “services” of the animated body schema which, on the other hand, can run its simulations if appropriately prompted by imitation and/or intention, with or without physical interaction. Successful experiences can leave a trace in the shared memory system and chunks of memory fragment may compete to participate to novel cooperative actions. And so on and so forth. At the heart of the system is lifelong training and learning but, different from the conventional learning paradigms in neural networks, where learning is somehow passively imposed by an external agent, in symbiotic robots there is an element of free choice of what is worth learning, driven by the interaction between the robot and the human partner. The proposed set of building blocks is certainly a rough approximation of what is needed by symbiotic robots but we believe it is a useful starting point for building a computational framework.

An Extended Passive Motion Paradigm for Human-Like Posture and Movement Planning in Redundant Manipulators

A major challenge in robotics and computational neuroscience is relative to the posture/movement problem in presence of kinematic redundancy. We recently addressed this issue using a principled approach which, in conjunction with nonlinear inverse optimization, allowed capturing postural strategies such as Donders' law. In this work, after presenting this general model specifying it as an extension of the Passive Motion Paradigm, we show how, once fitted to capture experimental postural strategies, the model is actually able to also predict movements. More specifically, the passive motion paradigm embeds two main intrinsic components: joint damping and joint stiffness. In previous work we showed that joint stiffness is responsible for static postures and, in this sense, its parameters are regressed to fit to experimental postural strategies. Here, we show how joint damping, in particular its anisotropy, directly affects task-space movements. Rather than using damping parameters to fit a posteriori task-space motions, we make the a priori hypothesis that damping is proportional to stiffness. This remarkably allows a postural-fitted model to also capture dynamic performance such as curvature and hysteresis of task-space trajectories during wrist pointing tasks, confirming and extending previous findings in literature.

Humanoid infers Archimedes' principle: understanding physical relations and object affordances through cumulative learning experiences

Emerging studies indicate that several species such as corvids, apes and children solve 'The Crow and the Pitcher' task (from Aesop's Fables) in diverse conditions. Hidden beneath this fascinating paradigm is a fundamental question: by cumulatively interacting with different objects, how can an agent abstract the underlying cause-effect relations to predict and creatively exploit potential affordances of novel objects in the context of sought goals? Re-enacting this Aesop's Fable task on a humanoid within an open-ended 'learning-prediction-abstraction' loop, we address this problem and (i) present a brain-guided neural framework that emulates rapid one-shot encoding of ongoing experiences into a long-term memory and (ii) propose four task-agnostic learning rules (elimination, growth, uncertainty and status quo) that correlate predictions from remembered past experiences with the unfolding present situation to gradually abstract the underlying causal relations. Driven by the proposed architecture, the ensuing robot behaviours illustrated causal learning and anticipation similar to natural agents. Results further demonstrate that by cumulatively interacting with few objects, the predictions of the robot in case of novel objects converge close to the physical law, i.e. the Archimedes principle: this being independent of both the objects explored during learning and the order of their cumulative exploration.

Task-space separation principle: a force-field approach to motion planning for redundant manipulators

In this work, we address human-like motor planning in redundant manipulators. Specifically, we want to capture postural synergies such as Donders' law, experimentally observed in humans during kinematically redundant tasks, and infer a minimal set of parameters to implement similar postural synergies in a kinematic model. For the model itself, although the focus of this paper is to solve redundancy by implementing postural strategies derived from experimental data, we also want to ensure that such postural control strategies do not interfere with other possible forms of motion control (in the task-space), i.e. solving the posture/movement problem. The redundancy problem is framed as a constrained optimization problem, traditionally solved via the method of Lagrange multipliers. The posture/movement problem can be tackled via the separation principle which, derived from experimental evidence, posits that the brain processes static torques (i.e. posture-dependent, such as gravitational torques) separately from dynamic torques (i.e. velocity-dependent). The separation principle has traditionally been applied at a joint torque level. Our main contribution is to apply the separation principle to Lagrange multipliers, which act as task-space force fields, leading to a task-space separation principle. In this way, we can separate postural control (implementing Donders' law) from various types of tasks-space movement planners. As an example, the proposed framework is applied to the (redundant) task of pointing with the human wrist. Nonlinear inverse optimization (NIO) is used to fit the model parameters and to capture motor strategies displayed by six human subjects during pointing tasks. The novelty of our NIO approach is that (i) the fitted motor strategy, rather than raw data, is used to filter and down-sample human behaviours; (ii) our framework is used to efficiently simulate model behaviour iteratively, until it converges towards the experimental human strategies.

Active inference and robot control: a case study

Active inference is a general framework for perception and action that is gaining prominence in computational and systems neuroscience but is less known outside these fields. Here, we discuss a proof-of-principle implementation of the active inference scheme for the control or the 7-DoF arm of a (simulated) PR2 robot. By manipulating visual and proprioceptive noise levels, we show under which conditions robot control under the active inference scheme is accurate. Besides accurate control, our analysis of the internal system dynamics (e.g. the dynamics of the hidden states that are inferred during the inference) sheds light on key aspects of the framework such as the quintessentially multimodal nature of control and the differential roles of proprioception and vision. In the discussion, we consider the potential importance of being able to implement active inference in robots. In particular, we briefly review the opportunities for modelling psychophysiological phenomena such as sensory attenuation and related failures of gain control, of the sort seen in Parkinson's disease. We also consider the fundamental difference between active inference and optimal control formulations, showing that in the former the heavy lifting shifts from solving a dynamical inverse problem to creating deep forward or generative models with dynamics, whose attracting sets prescribe desired behaviours.

Embodied Language Learning and Cognitive Bootstrapping: Methods and Design Principles

Co-development of action, conceptualization and social interaction mutually scaffold and support each other within a virtuous feedback cycle in the development of human language in children. Within this framework, the purpose of this article is to bring together diverse but complementary accounts of research methods that jointly contribute to our understanding of cognitive development and in particular, language acquisition in robots. Thus, we include research pertaining to developmental robotics, cognitive science, psychology, linguistics and neuroscience, as well as practical computer science and engineering. The different studies are not at this stage all connected into a cohesive whole; rather, they are presented to illuminate the need for multiple different approaches that complement each other in the pursuit of understanding cognitive development in robots. Extensive experiments involving the humanoid robot iCub are reported, while human learning relevant to developmental robotics has also contributed useful results.

Disparate approaches are brought together via common underlying design principles. Without claiming to model human language acquisition directly, we are nonetheless inspired by analogous development in humans and consequently, our investigations include the parallel co-development of action, conceptualization and social interaction. Though these different approaches need to ultimately be integrated into a coherent, unified body of knowledge, progress is currently also being made by pursuing individual methods.

Revisiting the body-schema concept in the context of whole-body postural-focal dynamics

The body-schema concept is revisited in the context of embodied cognition, further developing the theory formulated by Marc Jeannerod that the motor system is part of a simulation network related to action, whose function is not only to shape the motor system for preparing an action (either overt or covert) but also to provide the self with information on the feasibility and the meaning of potential actions. The proposed computational formulation is based on a dynamical system approach, which is linked to an extension of the equilibrium-point hypothesis, called Passive Motor Paradigm: this dynamical system generates goal-oriented, spatio-temporal, sensorimotor patterns, integrating a direct and inverse internal model in a multi-referential framework. The purpose of such computational model is to operate at the same time as a general synergy formation machinery for planning whole-body actions in humanoid robots and/or for predicting coordinated sensory-motor patterns in human movements. In order to illustrate the computational approach, the integration of simultaneous, even partially conflicting tasks will be analyzed in some detail with regard to postural-focal dynamics, which can be defined as the fusion of a focal task, namely reaching a target with the whole-body, and a postural task, namely maintaining overall stability.

A Neural Framework for Organization and Flexible Utilization of Episodic Memory in Cumulatively Learning Baby Humanoids

Cumulatively developing robots offer a unique opportunity to reenact the constant interplay between neural mechanisms related to learning, memory, prospection, and abstraction from the perspective of an integrated system that acts, learns, remembers, reasons, and makes mistakes. Situated within such interplay lie some of the computationally elusive and fundamental aspects of cognitive behavior: the ability to recall and flexibly exploit diverse experiences of one’s past in the context of the present to realize goals, simulate the future, and keep learning further. This article is an adventurous exploration in this direction using a simple engaging scenario of how the humanoid iCub learns to construct the tallest possible stack given an arbitrary set of objects to play with. The learning takes place cumulatively, with the robot interacting with different objects (some previously experienced, some novel) in an open-ended fashion. Since the solution itself depends on what objects are available in the “now,” multiple episodes of past experiences have to be remembered and creatively integrated in the context of the present to be successful. Starting from zero, where the robot knows nothing, we explore the computational basis of organization episodic memory in a cumulatively learning humanoid and address (1) how relevant past experiences can be reconstructed based on the present context, (2) how multiple stored episodic memories compete to survive in the neural space and not be forgotten, (3) how remembered past experiences can be combined with explorative actions to learn something new, and (4) how multiple remembered experiences can be recombined to generate novel behaviors (without exploration). Through the resulting behaviors of the robot as it builds, breaks, learns, and remembers, we emphasize that mechanisms of episodic memory are fundamental design features necessary to enable the survival of autonomous robots in a real world where neither everything can be known nor can everything be experienced.

Strategy switching in the stabilization of unstable dynamics

In order to understand mechanisms of strategy switching in the stabilization of unstable dynamics, this work investigates how human subjects learn to become skilled users of an underactuated bimanual tool in an unstable environment. The tool, which consists of a mass and two hand-held non-linear springs, is affected by a saddle-like force-field. The non-linearity of the springs allows the users to determine size and orientation of the tool stiffness ellipse, by using different patterns of bimanual coordination: minimal stiffness occurs when the two spring terminals are aligned and stiffness size grows by stretching them apart. Tool parameters were set such that minimal stiffness is insufficient to provide stable equilibrium whereas asymptotic stability can be achieved with sufficient stretching, although at the expense of greater effort. As a consequence, tool users have two possible strategies for stabilizing the mass in different regions of the workspace: 1) high stiffness feedforward strategy, aiming at asymptotic stability and 2) low stiffness positional feedback strategy aiming at bounded stability. The tool was simulated by a bimanual haptic robot with direct torque control of the motors. In a previous study we analyzed the behavior of naïve users and we found that they spontaneously clustered into two groups of approximately equal size. In this study we trained subjects to become expert users of both strategies in a discrete reaching task. Then we tested generalization capabilities and mechanism of strategy-switching by means of stabilization tasks which consist of tracking moving targets in the workspace. The uniqueness of the experimental setup is that it addresses the general problem of strategy-switching in an unstable environment, suggesting that complex behaviors cannot be explained in terms of a global optimization criterion but rather require the ability to switch between different sub-optimal mechanisms.

Computing reaching dynamics in motor cortex with Cartesian spatial coordinates

How neurons in the primary motor cortex control arm movements is not yet understood. Here we show that the equations of motion governing reaching simplify when expressed in spatial coordinates. In this fixed reference frame, joint torques are the sums of vector cross products between the spatial positions of limb segments and their spatial accelerations and velocities. The consequences that follow from this model explain many properties of neurons in the motor cortex, including directional broad, cosinelike tuning, nonuniformly distributed preferred directions dependent on the workspace, and the rotation of the population vector during arm movements. Remarkably, the torques can be directly computed as a linearly weighted sum of responses from cortical motoneurons, and the muscle tensions can be obtained as rectified linear sums of the joint torques. This allows the required muscle tensions to be computed rapidly from a trajectory in space with a feedforward network model.