Analysis of impact of limb segment length variations during reinforcement learning in four-legged robot

Crawling robots are becoming increasingly prevalent in both industrial and private applications. Despite their many advantages over other robot types, they have complex movement mechanics. Artificial intelligence can simplify this by reinforcement learning. This process requires configuring the training environment and defining input parameters, including a robot model for movement training. To translate the virtual results into real-world scenarios, a 3D model with appropriate mechanical parameters must be developed.These parameters can vary significantly between multiple mechanical configurations, which will further impact the reinforcement learning process of such a robot. For this reason, it was decided to test which limb configurations would work best in this process. Initially, various kinematic types of walking robots were analysed, drawing on the anatomy of mammals, reptiles, and insects for the biological model. The reptilian model was chosen for its balance of stability, dynamics, and energy efficiency. The article reviews the preparation of robot models and the configuration of the Unity3D development environment using the ML-Agents toolkit. The experiment examined how different limb lengths affect training, resulting in movement algorithms for various quadruped robot configurations using artificial neural networks. Based on the numerical results, the best configuration was the default, with the same length of the tibia as the thigh, achieving a reward function value of 883.9 and an episode length of 245.5. Taking into account the same criteria, the least efficient configuration was definitely the one characterised by the shortest thigh and the longest tibia among those considered. In its case, the reward function reached a value of only 526.2 with an episode lasting 999.0, which means that it never achieved the intended goal.

Implementation of ANN-Based Auto-Adjustable for a Pneumatic Servo System Embedded on FPGA

Artificial intelligence techniques for pneumatic robot manipulators have become of deep interest in industrial applications, such as non-high voltage environments, clean operations, and high power-to-weight ratio tasks. The principal advantages of this type of actuator are the implementation of clean energies, low cost, and easy maintenance. The disadvantages of working with pneumatic actuators are that they have non-linear characteristics. This paper proposes an intelligent controller embedded in a programmable logic device to minimize the non-linearities of the air behavior into a 3-degrees-of-freedom robot with pneumatic actuators. In this case, the device is suitable due to several electric valves, direct current motors signals, automatic controllers, and several neural networks. For every degree of freedom, three neurons adjust the gains for each controller. The learning process is constantly tuning the gain value to reach the minimum of the mean square error. Results plot a more appropriate behavior for a transitive time when the neurons work with the automatic controllers with a minimum mean error of ±1.2 mm.

A Seven-degrees-of-freedom Robot-arm Driven by Pneumatic Artificial Muscles for Humanoid Robots

Braided pneumatic artificial muscles, and in particular the better known type with a double helical braid usually called the McKibben muscle, seem to be at present the best means for motorizing robot-arms with artificial muscles. Their ability to develop high maximum force associated with lightness and a compact cylindrical shape, as well as their analogical behavior with natural skeletal muscle were very well emphasized in the 1980s by the development of the Bridgestone “soft robot” actuated by “rubbertuators”. Recent publications have presented ways for modeling McKibben artificial muscle as well as controlling its highly non-linear dynamic behavior. However, fewer studies have concentrated on analyzing the integration of artificial muscles with robot-arm architectures since the first Bridgestone prototypes were designed. In this paper we present the design of a 7R anthropomorphic robot-arm entirely actuated by antagonistic McKibben artificial muscle pairs. The validation of the robot-arm architecture was performed in a teleoperation mode.

A NEW APPROACH OF THE 2-AXES PAM ROBOT ARM IDENTIFICATION USING NEURAL MIMO NARX MODEL

In this paper, a novel MIMO Neural NARX model is used for simultaneously modeling and identifying both joints of the 2-axes PAM robot arm's inverse and forward dynamic model. The highly nonlinear cross effect of both links of the 2-axes PAM robot arm are thoroughly modeled through an Inverse and Forward Neural MIMO NARX Model-based identification process using experimental input-output training data. Consequently the proposed Inverse and Forward Neural MIMO NARX model scheme of the nonlinear 2-axes PAM robot arm has been investigated. The results show that the novel Inverse and Forward Neural MIMO NARX Model trained by Back Propagation learning algorithm yields outstanding performance and perfect accuracy.

A Cognitive Model for Generalization during Sequential Learning

Traditional artificial neural network models of learning suffer fromcatastrophic interference. They are commonly trained to perform only one specific task, and, when trained on a new task, they forget the original task completely. It has been shown that the foundational neurocomputational principles embodied by the Leabra cognitive modeling framework, specifically fast lateral inhibition and a local synaptic plasticity model that incorporates both correlational and error-based components, are sufficient to largely overcome this limitation during the sequential learning of multiple motor skills. Evidence has also provided that Leabra is able to generalize the subsequences of motor skills, when doing so is appropriate. In this paper, we provide a detailed analysis of the extent of generalization possible with Leabra during sequential learning of multiple tasks. For comparison, we measure the generalization exhibited by the backpropagation of error learning algorithm. Furthermore, we demonstrate the applicability of sequential learning to a pair of movement tasks using a simulated robotic arm.

Cerebellar-inspired adaptive control of a robot eye actuated by pneumatic artificial muscles

In this paper, a model of cerebellar function is implemented and evaluated in the control of a robot eye actuated by pneumatic artificial muscles. The investigated control problem is stabilization of the visual image in response to disturbances. This is analogous to the vestibuloocular reflex (VOR) in humans. The cerebellar model is structurally based on the adaptive filter, and the learning rule is computationally analogous to least-mean squares, where parameter adaptation at the parallel fiber/Purkinje cell synapse is driven by the correlation of the sensory error signal (carried by the climbing fiber) and the motor command signal. Convergence of the algorithm is first analyzed in simulation on a model of the robot and then tested online in both one and two degrees of freedom. The results show that this model of neural function successfully works on a real-world problem, providing empirical evidence for validating: 1) the generic cerebellar learning algorithm; 2) the function of the cerebellum in the VOR; and 3) the signal transmission between functional neural components of the VOR.

Proxy-based Sliding Mode Control of a Planar Pneumatic Manipulator

For a robotic system that shares its workspace with humans and physically interacts with them, safety is of paramount importance. In order to build a safe system, safety has to be considered in both hardware and software (control). In this paper, we present the safe control of a two-degree-of-freedom planar manipulator actuated by Pleated Pneumatic Artificial Muscles. Owing to its low weight and inherent compliance, the system hardware has excellent safety characteristics. In traditional control methods, safety and good tracking are often impossible to combine. This is different in the case of Proxy-Based Sliding Mode Control (PSMC), a novel control method introduced by Kikuuwe and Fujimoto. PSMC combines responsive and accurate tracking during normal operation with smooth, slow and safe recovery from large position errors. It can also make the system behave compliantly to external disturbances. We present both task- and joint-space implementations of PSMC applied to the pneumatic manipulator, and compare their performance with PID control. Good tracking results are obtained, especially with the joint-space implementation. Safety is evaluated by means of the Head Injury Criterion and by the maximum interaction force in the case of collision. It is found that in spite of the hardware safety features, the system is unsafe when under PID control. PSMC, on the other hand, provides increased safety as well as good tracking.

Collaborative Assembly Operation between Two Modular Robots Based on the Optical Position Feedback

This paper studies the cooperation between two master-slave modular robots. A cooperative robot system is set up with two modular robots and a dynamic optical meter-Optotrak. With Optotrak, the positions of the end effectors are measured as the optical position feedback, which is used to adjust the robots' end positions. A tri-layered motion controller is designed for the two cooperative robots. The RMRC control method is adopted to adjust the master robot to the desired position. With the kinematics constraints of the two robots including position and pose, joint velocity, and acceleration constraints, the two robots can cooperate well. A bolt and nut assembly experiment is executed to verify the methods.

Modeling and Control Considerations for Powered Lower-Limb Orthoses: A Design Study for Assisted STS

Lower-limb orthotic devices may be used to aid or restore mobility to the impaired user. Powered orthoses, in particular, hold great potential in improving the quality of life for individuals with locomotor difficulties because active control of an orthosis can aid limb movement in common tasks that may even be impossible if unaided. However, these devices have primarily remained the products of research labs with the number of effective commercial applications for the laity being nearly nonexistent. This paper provides an overview of the current status of powered orthoses and goes on to discuss key issues in modeling and control of powered orthoses so that designers can have a unified framework in developing user-oriented devices. Key concepts are demonstrated for a powered knee-orthosis intended for assisting the sit-to-stand task, and both pneumatic muscle and dc motor actuators are considered in this conceptual design study. In the final analysis, we conclude that the ability to provide sit-to-stand assistance is profoundly dependent on the type of control signal employed to control the actuator from the user–orthosis interface.

A two-input sliding-mode controller for a planar arm actuated by four pneumatic muscle groups

Multiple-input sliding-mode techniques are applied to a planar arm actuated by four groups of pneumatic muscle (PM) actuators in opposing pair configuration. The control objective is end-effector tracking of a desired path in Cartesian space. The inputs to the system are commanded input pressure differentials for the two opposing PM groups. An existing model for the muscle is incorporated into the arm equations of motion to arrive at a two-input, two-output nonlinear model of the planar arm that is affine in the input and, therefore, suitable for sliding-mode techniques. Relationships between static input pressures are derived for suitable arm behavior in the absence of a control signal. Simulation studies are reported.

Discrete time neural network synthesis using input and output activation functions

A new very fast algorithm for synthesis of a new structure of discrete-time neural networks (NN) is proposed. For this purpose the following concepts are employed: (i) combination of input and output activation functions, (ii) input time-varying signal distribution, (iii) time-discrete domain synthesis and (iv) one-step learning iteration approach. The problem of input-output mappings of time-varying vectors is solved. Simulation results based on the synthesis of a new structure of feedforward NN of an universal logical unit are presented. The proposed NN synthesis procedure is useful for applications to identification and control of nonlinear, very fast, dynamical systems. In this sense a feedforward NN for an adaptive nonlinear robot control is designed. Finally, a new algorithm for the direct inverse modeling of input/output nonquadratic systems is discussed.