Goal-Oriented Obstacle Avoidance with Deep Reinforcement Learning in Continuous Action Space

The Impact of LiDAR Configuration on Goal-Based Navigation within a Deep Reinforcement Learning Framework

Over the years, deep reinforcement learning (DRL) has shown great potential in mapless autonomous robot navigation and path planning. These DRL methods rely on robots equipped with different light detection and range (LiDAR) sensors with a wide field of view (FOV) configuration to perceive their environment. These types of LiDAR sensors are expensive and are not suitable for small-scale applications. In this paper, we address the performance effect of the LiDAR sensor configuration in DRL models. Our focus is on avoiding static obstacles ahead. We propose a novel approach that determines an initial FOV by calculating an angle of view using the sensor's width and the minimum safe distance required between the robot and the obstacle. The beams returned within the FOV, the robot's velocities, the robot's orientation to the goal point, and the distance to the goal point are used as the input state to generate new velocity values as the output action of the DRL. The cost function of collision avoidance and path planning is defined as the reward of the DRL model. To verify the performance of the proposed method, we adjusted the proposed FOV by ±10° giving a narrower and wider FOV. These new FOVs are trained to obtain collision avoidance and path planning DRL models to validate the proposed method. Our experimental setup shows that the LiDAR configuration with the computed angle of view as its FOV performs best with a success rate of 98% and a lower time complexity of 0.25 m/s. Additionally, using a Husky Robot, we demonstrate the model's good performance and applicability in the real world.

Leveraging Expert Demonstration Features for Deep Reinforcement Learning in Floor Cleaning Robot Navigation

In this paper, a Deep Reinforcement Learning (DRL)-based approach for learning mobile cleaning robot navigation commands that leverage experience from expert demonstrations is presented. First, expert demonstrations of robot motion trajectories in simulation in the cleaning robot domain are collected. The relevant motion features with regard to the distance to obstacles and the heading difference towards the navigation goal are extracted. Each feature weight is optimized with respect to the collected data, and the obtained values are assumed as representing the optimal motion of the expert navigation. A reward function is created based on the feature values to train a policy with semi-supervised DRL, where an immediate reward is calculated based on the closeness to the expert navigation. The presented results show the viability of this approach with regard to robot navigation as well as the reduced training time.

A Few-Shot Learning-Based Reward Estimation for Mapless Navigation of Mobile Robots Using a Siamese Convolutional Neural Network

Deep reinforcement learning-based approaches to mapless navigation have relied on the distance to the goal state being known a priori or that the distance to the goal can be obtained at each timestep. In artificial or simulated environments, obtaining the distance to the goal is considered a trivial task. Still, when applied to a real-world scenario, the distance must be obtained through complex localization techniques, and the use of localization techniques increases the complexity of the agent design. However, for agents navigating in unknown environments, using information about the goal to either form part of the state representation or act as the reward mechanism is usually expensive for both the robot design and for computing costs. This paper proposes using a pre-trained Siamese convolutional neural network (SCNN) to estimate the distance between an agent and its goal, thus enabling agents equipped with onboard cameras to navigate an unknown environment without needing localization sensors. This technique can be applied to environments where a goal location may be unknown, and the only information regarding the goal maybe a description of the goal state. Our experiments show that the Siamese network can learn the distance between the agent and its goal using relatively few training samples. Therefore, it is useful for mapless navigation using only visual state information and reduces the need for complex localization techniques.

Adaptive Simplex Architecture for Safe, Real-Time Robot Path Planning

The paper addresses the problem of using machine learning in practical robot applications, like dynamic path planning with obstacle avoidance, so as to achieve the performance level of machine learning model scorers in terms of speed and reliability, and the safety and accuracy level of possibly slower, exact algorithmic solutions to the same problems. To this end, the existing simplex architecture for safety assurance in critical systems is extended by an adaptation mechanism, in which one of the redundant controllers (called a high-performance controller) is represented by a trained machine learning model. This model is retrained using field data to reduce its failure rate and redeployed continuously. The proposed adaptive simplex architecture (ASA) is evaluated on the basis of a robot path planning application with dynamic obstacle avoidance in the context of two human-robot collaboration scenarios in manufacturing. The evaluation results indicate that ASA enables a response by the robot in real time when it encounters an obstacle. The solution predicted by the model is economic in terms of path length and smoother than analogous algorithmic solutions. ASA ensures safety by providing an acceptance test, which checks whether the predicted path crosses the obstacle; in which case a suboptimal, yet safe, solution is used.