Brain-inspired biomimetic robot control: a review

Complex robotic systems, such as humanoid robot hands, soft robots, and walking robots, pose a challenging control problem due to their high dimensionality and heavy non-linearities. Conventional model-based feedback controllers demonstrate robustness and stability but struggle to cope with the escalating system design and tuning complexity accompanying larger dimensions. In contrast, data-driven methods such as artificial neural networks excel at representing high-dimensional data but lack robustness, generalization, and real-time adaptiveness. In response to these challenges, researchers are directing their focus to biological paradigms, drawing inspiration from the remarkable control capabilities inherent in the human body. This has motivated the exploration of new control methods aimed at closely emulating the motor functions of the brain given the current insights in neuroscience. Recent investigation into these Brain-Inspired control techniques have yielded promising results, notably in tasks involving trajectory tracking and robot locomotion. This paper presents a comprehensive review of the foremost trends in biomimetic brain-inspired control methods to tackle the intricacies associated with controlling complex robotic systems.

Towards Multi-Objective Object Push-Grasp Policy Based on Maximum Entropy Deep Reinforcement Learning under Sparse Rewards

In unstructured environments, robots need to deal with a wide variety of objects with diverse shapes, and often, the instances of these objects are unknown. Traditional methods rely on training with large-scale labeled data, but in environments with continuous and high-dimensional state spaces, the data become sparse, leading to weak generalization ability of the trained models when transferred to real-world applications. To address this challenge, we present an innovative maximum entropy Deep Q-Network (ME-DQN), which leverages an attention mechanism. The framework solves complex and sparse reward tasks through probabilistic reasoning while eliminating the trouble of adjusting hyper-parameters. This approach aims to merge the robust feature extraction capabilities of Fully Convolutional Networks (FCNs) with the efficient feature selection of the attention mechanism across diverse task scenarios. By integrating an advantage function with the reasoning and decision-making of deep reinforcement learning, ME-DQN propels the frontier of robotic grasping and expands the boundaries of intelligent perception and grasping decision-making in unstructured environments. Our simulations demonstrate a remarkable grasping success rate of 91.6%, while maintaining excellent generalization performance in the real world.

Integrating Reinforcement Learning and Monte Carlo Tree Search for enhanced neoantigen vaccine design

Recent advances in cancer immunotherapy have highlighted the potential of neoantigen-based vaccines. However, the design of such vaccines is hindered by the possibility of weak binding affinity between the peptides and the patient's specific human leukocyte antigen (HLA) alleles, which may not elicit a robust adaptive immune response. Triggering cross-immunity by utilizing peptide mutations that have enhanced binding affinity to target HLA molecules, while preserving their homology with the original one, can be a promising avenue for neoantigen vaccine design. In this study, we introduced UltraMutate, a novel algorithm that combines Reinforcement Learning and Monte Carlo Tree Search, which identifies peptide mutations that not only exhibit enhanced binding affinities to target HLA molecules but also retains a high degree of homology with the original neoantigen. UltraMutate outperformed existing state-of-the-art methods in identifying affinity-enhancing mutations in an independent test set consisting of 3660 peptide-HLA pairs. UltraMutate further showed its applicability in the design of peptide vaccines for Human Papillomavirus and Human Cytomegalovirus, demonstrating its potential as a promising tool in the advancement of personalized immunotherapy.

Advancements in Learning-Based Navigation Systems for Robotic Applications in MRO Hangar: Review

The field of learning-based navigation for mobile robots is experiencing a surge of interest from research and industry sectors. The application of this technology for visual aircraft inspection tasks within a maintenance, repair, and overhaul (MRO) hangar necessitates efficient perception and obstacle avoidance capabilities to ensure a reliable navigation experience. The present reliance on manual labour, static processes, and outdated technologies limits operation efficiency in the inherently dynamic and increasingly complex nature of the real-world hangar environment. The challenging environment limits the practical application of conventional methods and real-time adaptability to changes. In response to these challenges, recent years research efforts have witnessed advancement with machine learning integration aimed at enhancing navigational capability in both static and dynamic scenarios. However, most of these studies have not been specific to the MRO hangar environment, but related challenges have been addressed, and applicable solutions have been developed. This paper provides a comprehensive review of learning-based strategies with an emphasis on advancements in deep learning, object detection, and the integration of multiple approaches to create hybrid systems. The review delineates the application of learning-based methodologies to real-time navigational tasks, encompassing environment perception, obstacle detection, avoidance, and path planning through the use of vision-based sensors. The concluding section addresses the prevailing challenges and prospective development directions in this domain.

Robot Programming from a Single Demonstration for High Precision Industrial Insertion

We propose a novel approach for robotic industrial insertion tasks using the Programming by Demonstration technique. Our method allows robots to learn a high-precision task by observing human demonstration once, without requiring any prior knowledge of the object. We introduce an Imitated-to-Finetuned approach that generates imitated approach trajectories by cloning the human hand's movements and then fine-tunes the goal position with a visual servoing approach. To identify features on the object used in visual servoing, we model object tracking as the moving object detection problem, separating each demonstration video frame into the moving foreground that includes the object and demonstrator's hand and the static background. Then a hand keypoints estimation function is used to remove the redundant features on the hand. The experiment shows that the proposed method can make robots learn precision industrial insertion tasks from a single human demonstration.

Effective Parametrization of Low Order Bézier Motion Primitives for Continuous-Curvature Path-Planning Applications

We propose a new parametrization of motion primitives based on Bézier curves that suits perfectly path-planning applications (and environment exploration) of wheeled mobile robots. The individual motion primitives can simply be calculated taking into account the requirements of path planning and the constraints of a vehicle, given in the form of the starting and ending orientations, velocities, turning rates, and curvatures. The proposed parametrization provides a natural geometric interpretation of the curve. The solution of the problem does not require optimization and is obtained by solving a system of simple polynomial equations. The resulting planar path composed of the primitives is guaranteed to be C2 continuous (the curvature is therefore continuous). The proposed primitives feature low order Bézier (third order polynomial) curves. This not only provides the final path with minimal required turns or unwanted oscillations that typically appear when using higher-order polynomial primitives due to Runge’s phenomenon but also makes the approach extremely computationally efficient. When used in path planning optimizers, the proposed primitives enable better convergence and conditionality of the optimization problem due to a low number of required parameters and a low order of the polynomials. The main contribution of the paper therefore lies in the analytic solution for the third-order Bézier motion primitive under given boundary conditions that guarantee continuous curvature of the composed spline path. The proposed approach is illustrated on some typical scenarios of path planning for wheeled mobile robots.

Dexterous Manipulation for Multi-Fingered Robotic Hands With Reinforcement Learning: A Review

With the increasing demand for the dexterity of robotic operation, dexterous manipulation of multi-fingered robotic hands with reinforcement learning is an interesting subject in the field of robotics research. Our purpose is to present a comprehensive review of the techniques for dexterous manipulation with multi-fingered robotic hands, such as the model-based approach without learning in early years, and the latest research and methodologies focused on the method based on reinforcement learning and its variations. This work attempts to summarize the evolution and the state of the art in this field and provide a summary of the current challenges and future directions in a way that allows future researchers to understand this field.

Position control of a planar cable-driven parallel robot using reinforcement learning

This study proposes a method based on reinforcement learning (RL) for point-to-point and dynamic reference position tracking control of a planar cable-driven parallel robots, which is a multi-input multi-output system (MIMO). The method eliminates the use of a tension distribution algorithm in controlling the system’s dynamics and inherently optimizes the cable tensions based on the reward function during the learning process. The deep deterministic policy gradient algorithm is utilized for training the RL agents in point-to-point and dynamic reference tracking tasks. The performances of the two agents are tested on their specifically trained tasks. Moreover, we also implement the agent trained for point-to-point tasks on the dynamic reference tracking and vice versa. The performances of the RL agents are compared with a classical PD controller. The results show that RL can perform quite well without the requirement of designing different controllers for each task if the system’s dynamics is learned well.

Smart Industrial Robot Control Trends, Challenges and Opportunities within Manufacturing

Industrial robots and associated control methods are continuously developing. With the recent progress in the field of artificial intelligence, new perspectives in industrial robot control strategies have emerged, and prospects towards cognitive robots have arisen. AI-based robotic systems are strongly becoming one of the main areas of focus, as flexibility and deep understanding of complex manufacturing processes are becoming the key advantage to raise competitiveness. This review first expresses the significance of smart industrial robot control in manufacturing towards future factories by listing the needs, requirements and introducing the envisioned concept of smart industrial robots. Secondly, the current trends that are based on different learning strategies and methods are explored. Current computer-vision, deep reinforcement learning and imitation learning based robot control approaches and possible applications in manufacturing are investigated. Gaps, challenges, limitations and open issues are identified along the way.

Artificial Intelligence Surgery: How Do We Get to Autonomous Actions in Surgery?

Most surgeons are skeptical as to the feasibility of autonomous actions in surgery. Interestingly, many examples of autonomous actions already exist and have been around for years. Since the beginning of this millennium, the field of artificial intelligence (AI) has grown exponentially with the development of machine learning (ML), deep learning (DL), computer vision (CV) and natural language processing (NLP). All of these facets of AI will be fundamental to the development of more autonomous actions in surgery, unfortunately, only a limited number of surgeons have or seek expertise in this rapidly evolving field. As opposed to AI in medicine, AI surgery (AIS) involves autonomous movements. Fortuitously, as the field of robotics in surgery has improved, more surgeons are becoming interested in technology and the potential of autonomous actions in procedures such as interventional radiology, endoscopy and surgery. The lack of haptics, or the sensation of touch, has hindered the wider adoption of robotics by many surgeons; however, now that the true potential of robotics can be comprehended, the embracing of AI by the surgical community is more important than ever before. Although current complete surgical systems are mainly only examples of tele-manipulation, for surgeons to get to more autonomously functioning robots, haptics is perhaps not the most important aspect. If the goal is for robots to ultimately become more and more independent, perhaps research should not focus on the concept of haptics as it is perceived by humans, and the focus should be on haptics as it is perceived by robots/computers. This article will discuss aspects of ML, DL, CV and NLP as they pertain to the modern practice of surgery, with a focus on current AI issues and advances that will enable us to get to more autonomous actions in surgery. Ultimately, there may be a paradigm shift that needs to occur in the surgical community as more surgeons with expertise in AI may be needed to fully unlock the potential of AIS in a safe, efficacious and timely manner.