Learning Compliant Box-in-Box Insertion through Haptic-Based Robotic Teleoperation

In modern logistics, the box-in-box insertion task is representative of a wide range of packaging applications, and automating compliant object insertion is difficult due to challenges in modelling the object deformation during insertion. Using Learning from Demonstration (LfD) paradigms, which are frequently used in robotics to facilitate skill transfer from humans to robots, can be one solution for complex tasks that are difficult to mathematically model. In order to automate the box-in-box insertion task for packaging applications, this study makes use of LfD techniques. The proposed framework has three phases. Firstly, a master-slave teleoperated robot system is used in the initial phase to haptically demonstrate the insertion task. Then, the learning phase involves identifying trends in the demonstrated trajectories using probabilistic methods, in this case, Gaussian Mixture Regression. In the third phase, the insertion task is generalised, and the robot adjusts to any object position using barycentric interpolation. This method is novel because it tackles tight insertion by taking advantage of the boxes' natural compliance, making it possible to complete the task even with a position-controlled robot. To determine whether the strategy is generalisable and repeatable, experimental validation was carried out.

DMPs-based skill learning for redundant dual-arm robotic synchronized cooperative manipulation

Dual-arm robot manipulation is applicable to many domains, such as industrial, medical, and home service scenes. Learning from demonstrations is a highly effective paradigm for robotic learning, where a robot learns from human actions directly and can be used autonomously for new tasks, avoiding the complicated analytical calculation for motion programming. However, the learned skills are not easy to generalize to new cases where special constraints such as varying relative distance limitation of robotic end effectors for human-like cooperative manipulations exist. In this paper, we propose a dynamic movement primitives (DMPs) based skills learning framework for redundant dual-arm robots. The method, with a coupling acceleration term to the DMPs function, is inspired by the transient performance control of Barrier Lyapunov Functions. The additional coupling acceleration term is calculated based on the constant joint distance and varying relative distance limitations of end effectors for object-approaching actions. In addition, we integrate the generated actions in joint space and the solution for a redundant dual-arm robot to complete a human-like manipulation. Simulations undertaken in Matlab and Gazebo environments certify the effectiveness of the proposed method.

Development and evaluation of demonstration information recording approach for wheelchair mounted robotic arm

To offer simple and convenient assistance for the elderly and disabled, researchers focus on programming by demonstration approach to improve the intelligence and adaptability of wheelchair mounted robotic arm assistive robot. But how to easily and quickly obtain the demonstration information is still an urgent problem to be solved. Based on the systematic analysis of the daily living tasks in need of robot assistance, this paper proposes the key-point-based programming by demonstration recording approach to quickly obtain the demonstration information and develops a specified demonstration interface to simplify the operation process. A corresponding evaluation approach is also proposed from the demonstration trajectories and demonstration process two aspects. Additionally, tasks of “holding water glass task”, “eating task”, and “opening door task” are carried out and experimental results, as well as comparative evaluations confirm the validity of the proposed approach with high efficiency. This study can not only offer a convenient and feasible way to obtain the demonstration information of daily living tasks, but also lay a good foundation for the assistive robot to learn relative motion skills, especially for the demonstrated dexterous manipulation skills, and semi-autonomously accomplish complex, multi-step tasks following the user’s instructions in the daily home environment.

Lower limb movement intention recognition for rehabilitation robot aided with projected recurrent neural network

For the lower limb rehabilitation robot, how to better realize intention recognition is the key issue in the practical application. Recognition of the patient’s movement intention is a challenging research work, which needs to be studied from the shallow to the deep. Specifically, it is necessary to ensure that the movement intention of the normal person can be accurately recognized, and then improve the model to realize the recognition of the movement intention of the patients. Therefore, before studying the patient’s movement intention, it is essential to consider the normal person first, which is also for safety considerations. In recent years, a new Hill-based muscle model has been demonstrated to be capable of directly estimating the joint angle intention in an open-loop form. On this basis, by introducing a recurrent neural network (RNN), the whole prediction process can achieve more accuracy in a closed-loop form. However, for the traditional RNN algorithms, the activation function must be convex, which brings some limitations to the solution of practical problems. Especially, when the convergence speed of the traditional RNN model is limited in the practical applications, as the error continues to decrease, the convergence performance of the traditional RNN model will be greatly affected. To this end, a projected recurrent neural network (PRNN) model is proposed, which relaxes the condition of the convex function and can be used in the saturation constraint case. In addition, the corresponding theoretical proof is given, and the PRNN method with saturation constraint has been successfully applied in the experiment of intention recognition of lower limb movement compared with the traditional RNN model.

Two-finger exoskeleton with force feedback for a mobile robot teleoperation

In this work, the design, manufacturing, instrumentation, and application of a two-finger exoskeleton with force feedback are presented. The exoskeleton is based on remote center of motion mechanisms in order to avoid mechanical interference with the user’s fingers and is manufactured by three-dimensional printing. The developed exoskeleton is applied in a mobile robot teleoperation by mapping the finger movements in forward and turning commands for the robot. The presence of obstacles detected by the robot is sensed by the user by means of a feedback force. The problem of simultaneously communicating a data acquisition card and the robot hardware by MATLAB ® Simulink® was solved by using an external Wi-Fi module. The result is a lightweight exoskeleton which is able to communicate bidirectionally with a mobile robot by a personal computer for teleoperation tasks. The success of the system implementation is proven by a set of experiments presented in the final part of the article.

Robust Real-Time Detection of Laparoscopic Instruments in Robot Surgery Using Convolutional Neural Networks with Motion Vector Prediction

More than half of post-operative complications can be prevented, and operation performances can be improved based on the feedback gathered from operations or notifications of the risks during operations in real time. However, existing surgical analysis methods are limited, because they involve time-consuming processes and subjective opinions. Therefore, the detection of surgical instruments is necessary for (a) conducting objective analyses, or (b) providing risk notifications associated with a surgical procedure in real time. We propose a new real-time detection algorithm for detection of surgical instruments using convolutional neural networks (CNNs). This algorithm is based on an object detection system YOLO9000 and ensures continuity of detection of the surgical tools in successive imaging frames based on motion vector prediction. This method exhibits a constant performance irrespective of a surgical instrument class, while the mean average precision (mAP) of all the tools is 84.7, with a speed of 38 frames per second (FPS).