Survey of imitation learning for robotic manipulation

Convolutional neural network model by deep learning and teaching robot in keyboard musical instrument teaching

Keyboard instruments play a significant role in the music teaching process, providing students with an enjoyable musical experience while enhancing their music literacy. This study aims to investigate the current state of keyboard instrument teaching in preschool education, identify existing challenges, and propose potential solutions using the literature review method. In response to identified shortcomings, this paper proposes integrating intelligent technology and subject teaching through the application of teaching robots in keyboard instrument education. Specifically, a Convolutional Neural Network model of Deep Learning is employed for system debugging, enabling the teaching robot to analyze students' images and movements during musical instrument play and deliver targeted teaching. Feedback from students who participated in keyboard instrument teaching with the robot indicates high satisfaction levels. This paper aims to diversify keyboard instruments' teaching mode, introduce the practical application of robots in classroom teaching, and facilitate personalized teaching catering to individual students' aptitudes.

Neurorehabilitation robotics: how much control should therapists have?

Robotic technologies for rehabilitating motor impairments from neurological injuries have been the focus of intensive research and capital investment for more than 30 years. However, these devices have failed to convincingly demonstrate greater restoration of patient function compared to conventional therapy. Nevertheless, robots have value in reducing the manual effort required for physical therapists to provide high-intensity, high-dose interventions. In most robotic systems, therapists remain outside the control loop to act as high-level supervisors, selecting and initiating robot control algorithms to achieve a therapeutic goal. The low-level physical interactions between the robot and the patient are handled by adaptive algorithms that can provide progressive therapy. In this perspective, we examine the physical therapist's role in the control of rehabilitation robotics and whether embedding therapists in lower-level robot control loops could enhance rehabilitation outcomes. We discuss how the features of many automated robotic systems, which can provide repeatable patterns of physical interaction, may work against the goal of driving neuroplastic changes that promote retention and generalization of sensorimotor learning in patients. We highlight the benefits and limitations of letting therapists physically interact with patients through online control of robotic rehabilitation systems, and explore the concept of trust in human-robot interaction as it applies to patient-robot-therapist relationships. We conclude by highlighting several open questions to guide the future of therapist-in-the-loop rehabilitation robotics, including how much control to give therapists and possible approaches for having the robotic system learn from therapist-patient interactions.

Biomimetic Approaches for Human Arm Motion Generation: Literature Review and Future Directions

In recent years, numerous studies have been conducted to analyze how humans subconsciously optimize various performance criteria while performing a particular task, which has led to the development of robots that are capable of performing tasks with a similar level of efficiency as humans. The complexity of the human body has led researchers to create a framework for robot motion planning to recreate those motions in robotic systems using various redundancy resolution methods. This study conducts a thorough analysis of the relevant literature to provide a detailed exploration of the different redundancy resolution methodologies used in motion generation for mimicking human motion. The studies are investigated and categorized according to the study methodology and various redundancy resolution methods. An examination of the literature revealed a strong trend toward formulating intrinsic strategies that govern human movement through machine learning and artificial intelligence. Subsequently, the paper critically evaluates the existing approaches and highlights their limitations. It also identifies the potential research areas that hold promise for future investigations.

Model-free inverse reinforcement learning with multi-intention, unlabeled, and overlapping demonstrations

In this paper, we define a novel inverse reinforcement learning (IRL) problem where the demonstrations are multi-intention, i.e., collected from multi-intention experts, unlabeled, i.e., without intention labels, and partially overlapping, i.e., shared between multiple intentions. In the presence of overlapping demonstrations, current IRL methods, developed to handle multi-intention and unlabeled demonstrations, cannot successfully learn the underlying reward functions. To solve this limitation, we propose a novel clustering-based approach to disentangle the observed demonstrations and experimentally validate its advantages. Traditional clustering-based approaches to multi-intention IRL, which are developed on the basis of model-based Reinforcement Learning (RL), formulate the problem using parametric density estimation. However, in high-dimensional environments and unknown system dynamics, i.e., model-free RL, the solution of parametric density estimation is only tractable up to the density normalization constant. To solve this, we formulate the problem as a mixture of logistic regressions to directly handle the unnormalized density. To research the challenges faced by overlapping demonstrations, we introduce the concepts of shared pair, which is a state-action pair that is shared in more than one intention, and separability, which resembles how well the multiple intentions can be separated in the joint state-action space. We provide theoretical analyses under the global optimality condition and the existence of shared pairs. Furthermore, we conduct extensive experiments on four simulated robotics tasks, extended to accept different intentions with specific levels of separability, and a synthetic driver task developed to directly control the separability. We evaluate the existing baselines on our defined problem and demonstrate, theoretically and experimentally, the advantages of our clustering-based solution, especially when the separability of the demonstrations decreases.

Smart Industrial Robot Control Trends, Challenges and Opportunities within Manufacturing

Industrial robots and associated control methods are continuously developing. With the recent progress in the field of artificial intelligence, new perspectives in industrial robot control strategies have emerged, and prospects towards cognitive robots have arisen. AI-based robotic systems are strongly becoming one of the main areas of focus, as flexibility and deep understanding of complex manufacturing processes are becoming the key advantage to raise competitiveness. This review first expresses the significance of smart industrial robot control in manufacturing towards future factories by listing the needs, requirements and introducing the envisioned concept of smart industrial robots. Secondly, the current trends that are based on different learning strategies and methods are explored. Current computer-vision, deep reinforcement learning and imitation learning based robot control approaches and possible applications in manufacturing are investigated. Gaps, challenges, limitations and open issues are identified along the way.

Service Robots: Trends and Technology

The 2021 sales volume in the market of service robots is attractive. Expert reports from the International Federation of Robotics confirm 27 billion USD in total market share. Moreover, the number of new startups with the denomination of service robots nowadays constitutes 29% of the total amount of robotic companies recorded in the United States. Those data, among other similar figures, remark the need for formal development in the service robots area, including knowledge transfer and literature reviews. Furthermore, the COVID-19 spread accelerated business units and some research groups to invest time and effort into the field of service robotics. Therefore, this research work intends to contribute to the formalization of service robots as an area of robotics, presenting a systematic review of scientific literature. First, a definition of service robots according to fundamental ontology is provided, followed by a detailed review covering technological applications; state-of-the-art, commercial technology; and application cases indexed on the consulted databases.

CNN-Based Classifier as an Offline Trigger for the CREDO Experiment

Gamification is known to enhance users' participation in education and research projects that follow the citizen science paradigm. The Cosmic Ray Extremely Distributed Observatory (CREDO) experiment is designed for the large-scale study of various radiation forms that continuously reach the Earth from space, collectively known as cosmic rays. The CREDO Detector app relies on a network of involved users and is now working worldwide across phones and other CMOS sensor-equipped devices. To broaden the user base and activate current users, CREDO extensively uses the gamification solutions like the periodical Particle Hunters Competition. However, the adverse effect of gamification is that the number of artefacts, i.e., signals unrelated to cosmic ray detection or openly related to cheating, substantially increases. To tag the artefacts appearing in the CREDO database we propose the method based on machine learning. The approach involves training the Convolutional Neural Network (CNN) to recognise the morphological difference between signals and artefacts. As a result we obtain the CNN-based trigger which is able to mimic the signal vs. artefact assignments of human annotators as closely as possible. To enhance the method, the input image signal is adaptively thresholded and then transformed using Daubechies wavelets. In this exploratory study, we use wavelet transforms to amplify distinctive image features. As a result, we obtain a very good recognition ratio of almost 99% for both signal and artefacts. The proposed solution allows eliminating the manual supervision of the competition process.

Learning for a Robot: Deep Reinforcement Learning, Imitation Learning, Transfer Learning

Dexterous manipulation of the robot is an important part of realizing intelligence, but manipulators can only perform simple tasks such as sorting and packing in a structured environment. In view of the existing problem, this paper presents a state-of-the-art survey on an intelligent robot with the capability of autonomous deciding and learning. The paper first reviews the main achievements and research of the robot, which were mainly based on the breakthrough of automatic control and hardware in mechanics. With the evolution of artificial intelligence, many pieces of research have made further progresses in adaptive and robust control. The survey reveals that the latest research in deep learning and reinforcement learning has paved the way for highly complex tasks to be performed by robots. Furthermore, deep reinforcement learning, imitation learning, and transfer learning in robot control are discussed in detail. Finally, major achievements based on these methods are summarized and analyzed thoroughly, and future research challenges are proposed.

Episodic Self-Imitation Learning with Hindsight

Episodic self-imitation learning, a novel self-imitation algorithm with a trajectory selection module and an adaptive loss function, is proposed to speed up reinforcement learning. Compared to the original self-imitation learning algorithm, which samples good state–action pairs from the experience replay buffer, our agent leverages entire episodes with hindsight to aid self-imitation learning. A selection module is introduced to filter uninformative samples from each episode of the update. The proposed method overcomes the limitations of the standard self-imitation learning algorithm, a transitions-based method which performs poorly in handling continuous control environments with sparse rewards. From the experiments, episodic self-imitation learning is shown to perform better than baseline on-policy algorithms, achieving comparable performance to state-of-the-art off-policy algorithms in several simulated robot control tasks. The trajectory selection module is shown to prevent the agent learning undesirable hindsight experiences. With the capability of solving sparse reward problems in continuous control settings, episodic self-imitation learning has the potential to be applied to real-world problems that have continuous action spaces, such as robot guidance and manipulation.