Learning-Based End-to-End Path Planning for Lunar Rovers with Safety Constraints

A Deep Learning Approach to Lunar Rover Global Path Planning Using Environmental Constraints and the Rover Internal Resource Status

This research proposes a novel approach to global path and resource planning for lunar rovers. The proposed method incorporates a range of constraints, including static, time-variant, and path-dependent factors related to environmental conditions and the rover's internal resource status. These constraints are integrated into a grid map as a penalty function, and a reinforcement learning-based framework is employed to address the resource constrained shortest path problem (RCSP). Compared to existing approaches referenced in the literature, our proposed method enables the simultaneous consideration of a broader spectrum of constraints. This enhanced flexibility leads to improved path search optimality. To evaluate the performance of our approach, this research applied the proposed learning architecture to lunar rover path search problems, generated based on real lunar digital elevation data. The simulation results demonstrate that our architecture successfully identifies a rover path while consistently adhering to user-defined environmental and rover resource safety criteria across all positions and time epochs. Furthermore, the simulation results indicate that our approach surpasses conventional methods that solely rely on environmental constraints.

Multi-UAV autonomous collision avoidance based on PPO-GIC algorithm with CNN-LSTM fusion network

This paper is concerned with the autonomous effective collision avoidance strategy for multiple unmanned aerial vehicles (multi-UAV) in limited airspace under the framework of proximal policy optimization (PPO) algorithm. An end-to-end deep reinforcement learning (DRL) control strategy and a potential-based reward function are designed. Next, the CNN-LSTM (CL) fusion network is constructed by fusing the convolutional neural network (CNN) and the long short-term memory network (LSTM), which realizes the feature interaction among the information of multi-UAV. Then, a generalized integral compensator (GIC) is introduced into the actor-critic structure, and the CLPPO-GIC algorithm is proposed by combining CL and GIC. Finally, we validate the learned policy in various simulation environments by performance evaluation. The simulation results show that the introduction of the LSTM network and GIC can further improve the efficiency of collision avoidance, and the robustness and accuracy of the algorithm are verified in different environments.

Path Planning Method of Mobile Robot Using Improved Deep Reinforcement Learning

A mobile robot path planning method based on improved deep reinforcement learning is proposed. First, in order to conform to the actual kinematics model of the robot, the continuous environmental state space and discrete action state space are designed. In addition, an improved deep Q-network (DQN) method is proposed, which takes the directly collected information as the training samples and combines the environmental state characteristics of the robot and the target point to be reached as the input of the network. DQN method takes the Q value at the current position as the output of the network model and uses $\epsilon$ -greedy strategy for action selection. Finally, the reward function combined with the artificial potential field method is designed to optimize the state-action space. The reward function solves the problem of sparse reward in the environmental state space and makes the action selection of the robot more accurate. Experiments show that compared with the classical DQN method, the average loss function value is reduced by 36.87% and the average reward value is increased by 12.96%, which can effectively improve the working efficiency of mobile robot.

A Deep Reinforcement Learning Strategy Combining Expert Experience Guidance for a Fruit-Picking Manipulator

When using deep reinforcement learning algorithms for path planning of a multi-DOF fruit-picking manipulator in unstructured environments, it is much too difficult for the multi-DOF manipulator to obtain high-value samples at the beginning of training, resulting in low learning and convergence efficiency. Aiming to reduce the inefficient exploration in unstructured environments, a reinforcement learning strategy combining expert experience guidance was first proposed in this paper. The ratios of expert experience to newly generated samples and the frequency of return visits to expert experience were studied by the simulation experiments. Some conclusions were that the ratio of expert experience, which declined from 0.45 to 0.35, was more effective in improving learning efficiency of the model than the constant ratio. Compared to an expert experience ratio of 0.35, the success rate increased by 1.26%, and compared to an expert experience ratio of 0.45, the success rate increased by 20.37%. The highest success rate was achieved when the frequency of return visits was 15 in 50 episodes, an improvement of 31.77%. The results showed that the proposed method can effectively improve the model performance and enhance the learning efficiency at the beginning of training in unstructured environments. This training method has implications for the training process of reinforcement learning in other domains.

Improved Path Planning for Indoor Patrol Robot Based on Deep Reinforcement Learning

To solve the problems of poor exploration ability and convergence speed of traditional deep reinforcement learning in the navigation task of the patrol robot under indoor specified routes, an improved deep reinforcement learning algorithm based on Pan/Tilt/Zoom(PTZ) image information was proposed in this paper. The obtained symmetric image information and target position information are taken as the input of the network, the speed of the robot is taken as the output of the next action, and the circular route with boundary is taken as the test. The improved reward and punishment function is designed to improve the convergence speed of the algorithm and optimize the path so that the robot can plan a safer path while avoiding obstacles first. Compared with Deep Q Network(DQN) algorithm, the convergence speed after improvement is shortened by about 40%, and the loss function is more stable.

Path Planning for Autonomous Mobile Robots: A Review

Providing mobile robots with autonomous capabilities is advantageous. It allows one to dispense with the intervention of human operators, which may prove beneficial in economic and safety terms. Autonomy requires, in most cases, the use of path planners that enable the robot to deliberate about how to move from its location at one moment to another. Looking for the most appropriate path planning algorithm according to the requirements imposed by users can be challenging, given the overwhelming number of approaches that exist in the literature. Moreover, the past review works analyzed here cover only some of these approaches, missing important ones. For this reason, our paper aims to serve as a starting point for a clear and comprehensive overview of the research to date. It introduces a global classification of path planning algorithms, with a focus on those approaches used along with autonomous ground vehicles, but is also extendable to other robots moving on surfaces, such as autonomous boats. Moreover, the models used to represent the environment, together with the robot mobility and dynamics, are also addressed from the perspective of path planning. Each of the path planning categories presented in the classification is disclosed and analyzed, and a discussion about their applicability is added at the end.

Quadrotor Autonomous Navigation in Semi-Known Environments Based on Deep Reinforcement Learning

In the application scenarios of quadrotors, it is expected that only part of the obstacles can be identified and located in advance. In order to make quadrotors fly safely in this situation, we present a deep reinforcement learning-based framework to realize autonomous navigation in semi-known environments. Specifically, the proposed framework utilizes the dueling double deep recurrent Q-learning, which can implement global path planning with the obstacle map as input. Moreover, the proposed framework combined with contrastive learning-based feature extraction can conduct real-time autonomous obstacle avoidance with monocular vision effectively. The experimental results demonstrate that our framework exhibits remarkable performance for both global path planning and autonomous obstacle avoidance.

A Grain-Scale Study of Mojave Mars Simulant (MMS-1)

Space exploration has attracted significant interest by government agencies and the scientific community in recent years in an attempt to explore possible scenarios of settling of facilities on the Moon and Mars surface. One of the important components in space exploration is related with the understanding of the geophysical and geotechnical characteristics of the surfaces of planets and their natural satellites and because of the limitation of available extra-terrestrial samples, many times researchers develop simulants, which mimic the properties and characteristics of the original materials. In the present study, characterization at the grain-scale was performed on the Mojave Mars Simulant (MMS-1) with emphasis on the frictional behavior of small size samples which follow the particle-to-particle configuration. Additional characterization was performed by means of surface composition and morphology analysis and the crushing behavior of individual grains. The results from the study present for the first time the micromechanical tribological response of Mars simulant, and attempts were also made to compare the behavior of this simulant with previously published results on other types of Earth and extra-terrestrial materials. Despite some similarities between Mars and Moon simulants, the unique characteristics of the MMS-1 samples resulted in significant differences and particularly in severe damage of the grain surfaces, which was also linked to the dilation behavior at the grain-scale.