A Deep Learning Approach for Biped Robot Locomotion Interface Using a Single Inertial Sensor

In this study, we introduce a novel framework that combines human motion parameterization from a single inertial sensor, motion synthesis from these parameters, and biped robot motion control using the synthesized motion. This framework applies advanced deep learning methods to data obtained from an IMU attached to a human subject's pelvis. This minimalistic sensor setup simplifies the data collection process, overcoming price and complexity challenges related to multi-sensor systems. We employed a Bi-LSTM encoder to estimate key human motion parameters: walking velocity and gait phase from the IMU sensor. This step is followed by a feedforward motion generator-decoder network that accurately produces lower limb joint angles and displacement corresponding to these parameters. Additionally, our method also introduces a Fourier series-based approach to generate these key motion parameters solely from user commands, specifically walking speed and gait period. Hence, the decoder can receive inputs either from the encoder or directly from the Fourier series parameter generator. The output of the decoder network is then utilized as a reference motion for the walking control of a biped robot, employing a constraint-consistent inverse dynamics control algorithm. This framework facilitates biped robot motion planning based on data from either a single inertial sensor or two user commands. The proposed method was validated through robot simulations in the MuJoco physics engine environment. The motion controller achieved an error of ≤5° in tracking the joint angles demonstrating the effectiveness of the proposed framework. This was accomplished using minimal sensor data or few user commands, marking a promising foundation for robotic control and human-robot interaction.

Investigation of Robot Expression Style in Human-Robot Interaction

In our daily conversation, we obtain considerable information from our interlocutor’s non-verbal behaviors, such as gaze and gestures. Several studies have shown that nonverbal messages are prominent factors in smoothing the process of human-robot interaction. Our previous studies have shown that not only a robot’s appearance but also its gestures, tone, and other nonverbal factors influence a person’s impression of it. The paper presented an analysis of the impressions made when human motions are implemented on a humanoid robot, and experiments were conducted to evaluate impressions made by robot expressions to analyze the sensations. The results showed the relation between robot expression patterns and human preferences. To further investigate biofeedback elicited by different robot styles of expression, a scenario-based experiment was done. The results revealed that people’s emotions can definitely be affected by robot behavior, and the robot’s way of expressing itself is what most influences whether or not it is perceived as friendly. The results show that it is potentially useful to combine our concept into a robot system to meet individual needs.

A topological navigation system for indoor environments based on perception events

The aim of the work presented in this article is to develop a navigation system that allows a mobile robot to move autonomously in an indoor environment using perceptions of multiple events. A topological navigation system based on events that imitates human navigation using sensorimotor abilities and sensorial events is presented. The increasing interest in building autonomous mobile systems makes the detection and recognition of perceptions a crucial task. The system proposed can be considered a perceptive navigation system as the navigation process is based on perception and recognition of natural and artificial landmarks, among others. The innovation of this work resides in the use of an integration interface to handle multiple events concurrently, leading to a more complete and advanced navigation system. The developed architecture enhances the integration of new elements due to its modularity and the decoupling between modules. Finally, experiments have been carried out in several mobile robots, and their results show the feasibility of the navigation system proposed and the effectiveness of the sensorial data integration managed as events.

Synthesis of New Dynamic Movement Primitives Through Search in a Hierarchical Database of Example Movements

This paper presents a novel approach to discovering motor primitives in a hierarchical database of example trajectories. An initial set of example trajectories is obtained by human demonstration. The trajectories are clustered and organized in a binary tree-like hierarchical structure, from which transition graphs at different levels of granularity are constructed. A novel procedure for searching in this hierarchical structure is presented. It can exploit the interdependencies between movements and can discover new series of partial paths. From these partial paths, complete new movements are generated by encoding them as dynamic movement primitives. In this way, the number of example trajectories that must be acquired with the assistance of a human teacher can be reduced. By combining the results of the hierarchical search with statistical generalization techniques, a complete representation of new, not directly demonstrated, movement primitives can be generated.

Statistical mutual conversion between whole body motion primitives and linguistic sentences for human motions

This paper describes a novel approach to linguistic mutual inference, which enables robots not only to linguistically interpret the motion patterns in the form of sentences but also to generate the motions from the sentences. The inference can be established based on two modules, the motion language model and the natural language model. The motion language model stochastically represents an association structure between symbols of motion patterns and the words in sentences assigned to the motion. This is a statistical model with a three layered structure of motion symbols, latent states and words. The natural language model statistically represents a structure of sentences based on word bigrams. The motion language model and the natural language model correspond to semantics and syntax respectively. An approach to the integration of motion language model with the natural language model allows the linguistic mutual inference for the robots. The two kinds of inference can be made by solving search problems, search for a sequence of words corresponding to a motion and search for a symbol of motion pattern corresponding to a sentence. The proposed approach to interpretation of motion patterns as sentences and generation of motion patterns from the sentences through the integration of motion language model with the natural language model is validated by an experiment on the human behavioral data.

Balance maintenance in high-speed motion of humanoid robot arm-based on the 6D constraints of momentum change rate

Based on the 6D constraints of momentum change rate (CMCR), this paper puts forward a real-time and full balance maintenance method for the humanoid robot during high-speed movement of its 7-DOF arm. First, the total momentum formula for the robot's two arms is given and the momentum change rate is defined by the time derivative of the total momentum. The author also illustrates the idea of full balance maintenance and analyzes the physical meaning of 6D CMCR and its fundamental relation to full balance maintenance. Moreover, discretization and optimization solution of CMCR has been provided with the motion constraint of the auxiliary arm's joint, and the solving algorithm is optimized. The simulation results have shown the validity and generality of the proposed method on the full balance maintenance in the 6 DOFs of the robot body under 6D CMCR. This method ensures 6D dynamics balance performance and increases abundant ZMP stability margin. The resulting motion of the auxiliary arm has large abundance in joint space, and the angular velocity and the angular acceleration of these joints lie within the predefined limits. The proposed algorithm also has good real-time performance.

FROM HUMAN MOTION CAPTURES TO HUMANOID SPATIAL COORDINATION

This paper describes a methodology translating human spatial coordination in a humanoid robots context. Once the human locomotion is captured, we highlight coordination relations describing motions. Relations and inverse kinematics are applied to a virtual humanoid, which is a tradeoff between human (anthropomorphic proportions) and robot (joints configuration). Further, we quantify all required adaptations for a real humanoid, such as scaling and equilibrium. Finally, this methodology is applied to a specific humanoid robot (called NAO) in order to illustrate and compare some preliminary results.

Reverse control for humanoid robot task recognition

Efficient methods to perform motion recognition have been developed using statistical tools. Those methods rely on primitive learning in a suitable space, for example, the latent space of the joint angle and/or adequate task spaces. Learned primitives are often sequential: A motion is segmented according to the time axis. When working with a humanoid robot, a motion can be decomposed into parallel subtasks. For example, in a waiter scenario, the robot has to keep some plates horizontal with one of its arms while placing a plate on the table with its free hand. Recognition can thus not be limited to one task per consecutive segment of time. The method presented in this paper takes advantage of the knowledge of what tasks the robot is able to do and how the motion is generated from this set of known controllers, to perform a reverse engineering of an observed motion. This analysis is intended to recognize parallel tasks that have been used to generate a motion. The method relies on the task-function formalism and the projection operation into the null space of a task to decouple the controllers. The approach is successfully applied on a real robot to disambiguate motion in different scenarios where two motions look similar but have different purposes.

From human motion capture to humanoid locomotion imitation Application to the robots HRP-2 and HOAP-3

This paper presents a method to generate humanoid gaits from a human locomotion pattern recorded by a motion capture system. Thirty seven reflective markers were fixed on the human subject skin in order to get the subject whole body motion. To reproduce the human gait, especially the toes and heel contacts, the front and back edges of the robot's feet are used as support at the start and the end of the double support phase. The balance of the robot is respected using the zero moment point (ZMP) criterion and confirmed by the simulation software OPENHRP (General Robotics, Inc®). First, the feet trajectory as well as the ZMP reference trajectory are defined from the motion of the robot controlled as a marionette with the measured human joint angles. Then a specific inverse kinematic (IK) algorithm is proposed to find the humanoid robot's joint trajectories respecting the constraints of balance, floor contacts, and joint limits. The studied motion presented in this paper is a human walking trajectory containing a start, a movement in a straight line, a stop, and a quarter turn. The method was developed to be easily used for human-like robots of different sizes, masses, and structures and has been tested on the robot HRP-2 (AIST, Kawada Industries, Inc®) and on the small-sized humanoid robot HOAP-3 (Fujitsu Automation Ltd®).