Deep Reinforcement Learning-Based Path Planning for Multi-Arm Manipulators with Periodically Moving Obstacles

A Deep Reinforcement Learning Strategy Combining Expert Experience Guidance for a Fruit-Picking Manipulator

When using deep reinforcement learning algorithms for path planning of a multi-DOF fruit-picking manipulator in unstructured environments, it is much too difficult for the multi-DOF manipulator to obtain high-value samples at the beginning of training, resulting in low learning and convergence efficiency. Aiming to reduce the inefficient exploration in unstructured environments, a reinforcement learning strategy combining expert experience guidance was first proposed in this paper. The ratios of expert experience to newly generated samples and the frequency of return visits to expert experience were studied by the simulation experiments. Some conclusions were that the ratio of expert experience, which declined from 0.45 to 0.35, was more effective in improving learning efficiency of the model than the constant ratio. Compared to an expert experience ratio of 0.35, the success rate increased by 1.26%, and compared to an expert experience ratio of 0.45, the success rate increased by 20.37%. The highest success rate was achieved when the frequency of return visits was 15 in 50 episodes, an improvement of 31.77%. The results showed that the proposed method can effectively improve the model performance and enhance the learning efficiency at the beginning of training in unstructured environments. This training method has implications for the training process of reinforcement learning in other domains.

Overcoming Challenges of Applying Reinforcement Learning for Intelligent Vehicle Control

Reinforcement learning (RL) is a booming area in artificial intelligence. The applications of RL are endless nowadays, ranging from fields such as medicine or finance to manufacturing or the gaming industry. Although multiple works argue that RL can be key to a great part of intelligent vehicle control related problems, there are many practical problems that need to be addressed, such as safety related problems that can result from non-optimal training in RL. For instance, for an RL agent to be effective it should first cover all the situations during training that it may face later. This is often difficult when applied to the real-world. In this work we investigate the impact of RL applied to the context of intelligent vehicle control. We analyse the implications of RL in path planning tasks and we discuss two possible approaches to overcome the gap between the theorical developments of RL and its practical applications. Specifically, firstly this paper discusses the role of Curriculum Learning (CL) to structure the learning process of intelligent vehicle control in a gradual way. The results show how CL can play an important role in training agents in such context. Secondly, we discuss a method of transferring RL policies from simulation to reality in order to make the agent experience situations in simulation, so it knows how to react to them in reality. For that, we use Arduino Yún controlled robots as our platforms. The results enhance the effectiveness of the presented approach and show how RL policies can be transferred from simulation to reality even when the platforms are resource limited.

A Novel Hybrid Path Planning Method Based on Q-Learning and Neural Network for Robot Arm

Path planning for robot arms to reach a target and avoid obstacles has had a crucial role in manufacturing automation. Although many path planning algorithms, including RRT, APF, PRM, and RL-based, have been presented, they have many problems: a time-consuming process, high computational costs, slowness, non-optimal paths, irregular paths, failure to find a path, and complexity. Scholars have tried to address some of these issues. However, those methods still suffer from slowness and complexity. In order to address these two limitations, this paper presents a new hybrid path planning method that contains two separate parts: action-finding (active approach) and angle-finding (passive approach). In the active phase, the Q-learning algorithm is used to find a sequence of simple actions, including up, down, left, and right, to reach the target cell in a gridded workspace. In the passive phase, the joints angles of the robot arm, with respect to the found actions, are obtained by the trained neural network. The simulation and test results show that this hybrid approach significantly improves the slowness and complexity due to using the simplified agent-environment interaction in the active phase and simple computing the joints angles in the passive phase.