Gesture recognition based on multilevel multimodal feature fusion

Grasping detection of dual manipulators based on Markov decision process with neural network

With the development of artificial intelligence, robots are widely used in various fields, grasping detection has been the focus of intelligent robot research. A dual manipulator grasping detection model based on Markov decision process is proposed to realize the stable grasping with complex multiple objects in this paper. Based on the principle of Markov decision process, the cross entropy convolutional neural network and full convolutional neural network are used to parameterize the grasping detection model of dual manipulators which are two-finger manipulator and vacuum sucker manipulator for multi-objective unknown objects. The data set generated in the simulated environment is used to train the two grasping detection networks. By comparing the grasping quality of the detection network output the best grasping by the two grasping methods, the network with better detection effect corresponding to the two grasping methods of two-finger and vacuum sucker is determined, and the dual manipulator grasping detection model is constructed in this paper. Robot grasping experiments are carried out, and the experimental results show that the proposed dual manipulator grasping detection method achieves 90.6% success rate, which is much higher than the other groups of experiments. The feasibility and superiority of the dual manipulator grasping detection method based on Markov decision process are verified.

Real-Time Target Detection Method Based on Lightweight Convolutional Neural Network

The continuous development of deep learning improves target detection technology day by day. The current research focuses on improving the accuracy of target detection technology, resulting in the target detection model being too large. The number of parameters and detection speed of the target detection model are very important for the practical application of target detection technology in embedded systems. This article proposed a real-time target detection method based on a lightweight convolutional neural network to reduce the number of model parameters and improve the detection speed. In this article, the depthwise separable residual module is constructed by combining depthwise separable convolution and non–bottleneck-free residual module, and the depthwise separable residual module and depthwise separable convolution structure are used to replace the VGG backbone network in the SSD network for feature extraction of the target detection model to reduce parameter quantity and improve detection speed. At the same time, the convolution kernels of 1 × 3 and 3 × 1 are used to replace the standard convolution of 3 × 3 by adding the convolution kernels of 1 × 3 and 3 × 1, respectively, to obtain multiple detection feature graphs corresponding to SSD, and the real-time target detection model based on a lightweight convolutional neural network is established by integrating the information of multiple detection feature graphs. This article used the self-built target detection dataset in complex scenes for comparative experiments; the experimental results verify the effectiveness and superiority of the proposed method. The model is tested on video to verify the real-time performance of the model, and the model is deployed on the Android platform to verify the scalability of the model.

Path Planning Optimization of Intelligent Vehicle Based on Improved Genetic and Ant Colony Hybrid Algorithm

Intelligent vehicles were widely used in logistics handling, agriculture, medical service, industrial production, and other industries, but they were often not smooth enough in planning the path, and the number of turns was large, resulting in high energy consumption. Aiming at the unsmooth path planning problem of four-wheel intelligent vehicle path planning algorithm, this article proposed an improved genetic and ant colony hybrid algorithm, and the physical model of intelligent vehicle was established. This article first improved ant colony optimization algorithm about heuristic function with the adaptive change of evaporation factor. Then, it improved the genetic algorithm on fitness function, adaptive adjustment of crossover factor, and mutation factor. Last, this article proposed the improved hybrid algorithm with the addition of a deletion operator, adoption of an elite retention strategy, and addition of suboptimal solutions obtained from the improved ant colony algorithm to improved genetic algorithm to obtain optimized new populations. The simulation environment for this article is windows 10, the processor is Intel Core i5-5257U, the running memory is 4GB, the compilation environment is MATLAB2018b, the number of ant samples is 50, the maximum number of iterations is 100, the initial population size of the genetic algorithm is 200, and the maximum number of iterations is 50. Simulation and physical experiments show that the improved hybrid algorithm is effective. Compared with the traditional hybrid algorithm, the improved hybrid algorithm reduced by 46% in the average number of iterations and 75% in the average number of turns in a simple grid. The improved hybrid algorithm reduced by 47% in the average number of iterations and 21% in the average number of turns in a complex grid. The improved hybrid algorithm works better to reduce the number of turns in simple maps.

Multi-Objective Optimization Design of Ladle Refractory Lining Based on Genetic Algorithm

Genetic algorithm is widely used in multi-objective mechanical structure optimization. In this paper, a genetic algorithm-based optimization method for ladle refractory lining structure is proposed. First, the parametric finite element model of the new ladle refractory lining is established by using ANSYS Workbench software. The refractory lining is mainly composed of insulating layer, permanent layer and working layer. Secondly, a mathematical model for multi-objective optimization is established to reveal the functional relationship between the maximum equivalent force on the ladle lining, the maximum temperature on the ladle shell, the total mass of the ladle and the structural parameters of the ladle refractory lining. Genetic algorithm translates the optimization process of ladle refractory lining into natural evolution and selection. The optimization results show that, compared with the unoptimized ladle refractory lining structure (insulation layer thickness of 0 mm, permanent layer thickness of 81 mm, and working layer thickness of 152 mm), the refractory lining with insulation layer thickness of 8.02 mm, permanent layer thickness of 76.20 mm, and working layer thickness of 148.61 mm has the best thermal insulation performance and longer service life within the variation of ladle refractory lining structure parameters. Finally, the results of the optimization are verified and analyzed in this paper. The study found that by optimizing the design of the ladle refractory lining, the maximum equivalent force on the ladle lining, the maximum temperature on the ladle shell and the ladle mass were reduced. The thermal insulation performance and the lightweight performance of the ladle are improved, which is very important for improving the service life of the ladle.

Robust Image Matching Based on Image Feature and Depth Information Fusion

In this paper, we propose a robust image feature extraction and fusion method to effectively fuse image feature and depth information and improve the registration accuracy of RGB-D images. The proposed method directly splices the image feature point descriptors with the corresponding point cloud feature descriptors to obtain the fusion descriptor of the feature points. The fusion feature descriptor is constructed based on the SIFT, SURF, and ORB feature descriptors and the PFH and FPFH point cloud feature descriptors. Furthermore, the registration performance based on fusion features is tested through the RGB-D datasets of YCB and KITTI. ORBPFH reduces the false-matching rate by 4.66~16.66%, and ORBFPFH reduces the false-matching rate by 9~20%. The experimental results show that the RGB-D robust feature extraction and fusion method proposed in this paper is suitable for the fusion of ORB with PFH and FPFH, which can improve feature representation and registration, representing a novel approach for RGB-D image matching.

Improved Multi-Stream Convolutional Block Attention Module for sEMG-Based Gesture Recognition

As a key technology for the non-invasive human-machine interface that has received much attention in the industry and academia, surface EMG (sEMG) signals display great potential and advantages in the field of human-machine collaboration. Currently, gesture recognition based on sEMG signals suffers from inadequate feature extraction, difficulty in distinguishing similar gestures, and low accuracy of multi-gesture recognition. To solve these problems a new sEMG gesture recognition network called Multi-stream Convolutional Block Attention Module-Gate Recurrent Unit (MCBAM-GRU) is proposed, which is based on sEMG signals. The network is a multi-stream attention network formed by embedding a GRU module based on CBAM. Fusing sEMG and ACC signals further improves the accuracy of gesture action recognition. The experimental results show that the proposed method obtains excellent performance on dataset collected in this paper with the recognition accuracies of 94.1%, achieving advanced performance with accuracy of 89.7% on the Ninapro DB1 dataset. The system has high accuracy in classifying 52 kinds of different gestures, and the delay is less than 300 ms, showing excellent performance in terms of real-time human-computer interaction and flexibility of manipulator control.

Multi-Scale Feature Fusion Convolutional Neural Network for Indoor Small Target Detection

The development of object detection technology makes it possible for robots to interact with people and the environment, but the changeable application scenarios make the detection accuracy of small and medium objects in the practical application of object detection technology low. In this paper, based on multi-scale feature fusion of indoor small target detection method, using the device to collect different indoor images with angle, light, and shade conditions, and use the image enhancement technology to set up and amplify a date set, with indoor scenarios and the SSD algorithm in target detection layer and its adjacent features fusion. The Faster R-CNN, YOLOv5, SSD, and SSD target detection models based on multi-scale feature fusion were trained on an indoor scene data set based on transfer learning. The experimental results show that multi-scale feature fusion can improve the detection accuracy of all kinds of objects, especially for objects with a relatively small scale. In addition, although the detection speed of the improved SSD algorithm decreases, it is faster than the Faster R-CNN, which better achieves the balance between target detection accuracy and speed.

A Tandem Robotic Arm Inverse Kinematic Solution Based on an Improved Particle Swarm Algorithm

The analysis of robot inverse kinematic solutions is the basis of robot control and path planning, and is of great importance for research. Due to the limitations of the analytical and geometric methods, intelligent algorithms are more advantageous because they can obtain approximate solutions directly from the robot’s positive kinematic equations, saving a large number of computational steps. Particle Swarm Algorithm (PSO), as one of the intelligent algorithms, is widely used due to its simple principle and excellent performance. In this paper, we propose an improved particle swarm algorithm for robot inverse kinematics solving. Since the setting of weights affects the global and local search ability of the algorithm, this paper proposes an adaptive weight adjustment strategy for improving the search ability. Considering the running time of the algorithm, this paper proposes a condition setting based on the limit joints, and introduces the position coefficient k in the velocity factor. Meanwhile, an exponential product form modeling method (POE) based on spinor theory is chosen. Compared with the traditional DH modeling method, the spinor approach describes the motion of a rigid body as a whole and avoids the singularities that arise when described by a local coordinate system. In order to illustrate the advantages of the algorithm in terms of accuracy, time, convergence and adaptability, three experiments were conducted with a general six-degree-of-freedom industrial robotic arm, a PUMA560 robotic arm and a seven-degree-of-freedom robotic arm as the research objects. In all three experiments, the parameters of the robot arm, the range of joint angles, and the initial attitude and position of the end-effector of the robot arm are given, and the attitude and position of the impact point of the end-effector are set to verify whether the joint angles found by the algorithm can reach the specified positions. In Experiments 2 and 3, the algorithm proposed in this paper is compared with the traditional particle swarm algorithm (PSO) and quantum particle swarm algorithm (QPSO) in terms of position and direction solving accuracy, operation time, and algorithm convergence. The results show that compared with the other two algorithms, the algorithm proposed in this paper can ensure higher position accuracy and orientation accuracy of the robotic arm end-effector. the position error of the algorithm proposed in this paper is 0 and the maximum orientation error is 1.29 × 10–8. while the minimum position error of the other two algorithms is −1.64 × 10–5 and the minimum orientation error is −4.03 × 10–6. In terms of operation time, the proposed algorithm in this paper has shorter operation time compared with the other two algorithms. In the last two experiments, the computing time of the proposed algorithm is 0.31851 and 0.30004s respectively, while the shortest computing time of the other two algorithms is 0.33359 and 0.30521s respectively. In terms of algorithm convergence, the proposed algorithm can achieve faster and more stable convergence than the other two algorithms. After changing the experimental subjects, the proposed algorithm still maintains its advantages in terms of accuracy, time and convergence, which indicates that the proposed algorithm is more applicable and has certain potential in solving the multi-arm inverse kinematics solution. This paper provides a new way of thinking for solving the multi-arm inverse kinematics solution problem.

Low-Illumination Image Enhancement Algorithm Based on Improved Multi-Scale Retinex and ABC Algorithm Optimization

In order to solve the problems of poor image quality, loss of detail information and excessive brightness enhancement during image enhancement in low light environment, we propose a low-light image enhancement algorithm based on improved multi-scale Retinex and Artificial Bee Colony (ABC) algorithm optimization in this paper. First of all, the algorithm makes two copies of the original image, afterwards, the irradiation component of the original image is obtained by used the structure extraction from texture via relative total variation for the first image, and combines it with the multi-scale Retinex algorithm to obtain the reflection component of the original image, which are simultaneously enhanced using histogram equalization, bilateral gamma function correction and bilateral filtering. In the next part, the second image is enhanced by histogram equalization and edge-preserving with Weighted Guided Image Filtering (WGIF). Finally, the weight-optimized image fusion is performed by ABC algorithm. The mean values of Information Entropy (IE), Average Gradient (AG) and Standard Deviation (SD) of the enhanced images are respectively 7.7878, 7.5560 and 67.0154, and the improvement compared to original image is respectively 2.4916, 5.8599 and 52.7553. The results of experiment show that the algorithm proposed in this paper improves the light loss problem in the image enhancement process, enhances the image sharpness, highlights the image details, restores the color of the image, and also reduces image noise with good edge preservation which enables a better visual perception of the image.

Time Optimal Trajectory Planing Based on Improved Sparrow Search Algorithm

Complete trajectory planning includes path planning, inverse solution solving and trajectory optimization. In this paper, a highly smooth and time-saving approach to trajectory planning is obtained by improving the kinematic and optimization algorithms for the time-optimal trajectory planning problem. By partitioning the joint space, the paper obtains an inverse solution calculation based on the partitioning of the joint space, saving 40% of the inverse kinematics solution time. This means that a large number of computational resources can be saved in trajectory planning. In addition, an improved sparrow search algorithm (SSA) is proposed to complete the solution of the time-optimal trajectory. A Tent chaotic mapping was used to optimize the way of generating initial populations. The algorithm was further improved by combining it with an adaptive step factor. The experiments demonstrated the performance of the improved SSA. The robot’s trajectory is further optimized in time by an improved sparrow search algorithm. Experimental results show that the method can improve convergence speed and global search capability and ensure smooth trajectories.

Attitude Stabilization Control of Autonomous Underwater Vehicle Based on Decoupling Algorithm and PSO-ADRC

Autonomous Underwater Vehicle are widely used in industries, such as marine resource exploitation and fish farming, but they are often subject to a large amount of interference which cause poor control stability, while performing their tasks. A decoupling control algorithm is proposed and A single control volume–single attitude angle model is constructed for the problem of severe coupling in the control system of attitude of six degrees of freedom Autonomous Underwater Vehicle. Aiming at the problem of complex Active Disturbance Rejection Control (ADRC) adjustment relying on manual experience, the PSO-ADRC algorithm is proposed to realize the automatic adjustment of its parameters, which improves the anti-interference ability and control accuracy of Autonomous Underwater Vehicle in dynamic environment. The anti-interference ability and control accuracy of the method were verified through experiments.

Self-Tuning Control of Manipulator Positioning Based on Fuzzy PID and PSO Algorithm

With the manipulator performs fixed-point tasks, it becomes adversely affected by external disturbances, parameter variations, and random noise. Therefore, it is essential to improve the robust and accuracy of the controller. In this article, a self-tuning particle swarm optimization (PSO) fuzzy PID positioning controller is designed based on fuzzy PID control. The quantization and scaling factors in the fuzzy PID algorithm are optimized by PSO in order to achieve high robustness and high accuracy of the manipulator. First of all, a mathematical model of the manipulator is developed, and the manipulator positioning controller is designed. A PD control strategy with compensation for gravity is used for the positioning control system. Then, the PID controller parameters dynamically are minute-tuned by the fuzzy controller 1. Through a closed-loop control loop to adjust the magnitude of the quantization factors–proportionality factors online. Correction values are outputted by the modified fuzzy controller 2. A quantization factor–proportion factor online self-tuning strategy is achieved to find the optimal parameters for the controller. Finally, the control performance of the improved controller is verified by the simulation environment. The results show that the transient response speed, tracking accuracy, and follower characteristics of the system are significantly improved.

Intelligent Detection of Steel Defects Based on Improved Split Attention Networks

The intelligent monitoring and diagnosis of steel defects plays an important role in improving steel quality, production efficiency, and associated smart manufacturing. The application of the bio-inspired algorithms to mechanical engineering problems is of great significance. The split attention network is an improvement of the residual network, and it is an improvement of the visual attention mechanism in the bionic algorithm. In this paper, based on the feature pyramid network and split attention network, the network is improved and optimised in terms of data enhancement, multi-scale feature fusion and network structure optimisation. The DF-ResNeSt50 network model is proposed, which introduces a simple modularized split attention block, which can improve the attention mechanism of cross-feature graph groups. Finally, experimental validation proves that the proposed network model has good performance and application prospects in the intelligent detection of steel defects.

Dynamic Gesture Recognition Using Surface EMG Signals Based on Multi-Stream Residual Network

Gesture recognition technology is widely used in the flexible and precise control of manipulators in the assisted medical field. Our MResLSTM algorithm can effectively perform dynamic gesture recognition. The result of surface EMG signal decoding is applied to the controller, which can improve the fluency of artificial hand control. Much current gesture recognition research using sEMG has focused on static gestures. In addition, the accuracy of recognition depends on the extraction and selection of features. However, Static gesture research cannot meet the requirements of natural human-computer interaction and dexterous control of manipulators. Therefore, a multi-stream residual network (MResLSTM) is proposed for dynamic hand movement recognition. This study aims to improve the accuracy and stability of dynamic gesture recognition. Simultaneously, it can also advance the research on the smooth control of the Manipulator. We combine the residual model and the convolutional short-term memory model into a unified framework. The architecture extracts spatiotemporal features from two aspects: global and deep, and combines feature fusion to retain essential information. The strategy of pointwise group convolution and channel shuffle is used to reduce the number of network calculations. A dataset is constructed containing six dynamic gestures for model training. The experimental results show that on the same recognition model, the gesture recognition effect of fusion of sEMG signal and acceleration signal is better than that of only using sEMG signal. The proposed approach obtains competitive performance on our dataset with the recognition accuracies of 93.52%, achieving state-of-the-art performance with 89.65% precision on the Ninapro DB1 dataset. Our bionic calculation method is applied to the controller, which can realize the continuity of human-computer interaction and the flexibility of manipulator control.

T2-fuzzy multi-fused facial image fusion (T2FMFImgF): An efficient face recognition

This paper presents a novel decision-making method for face recognition where the features were extracted from the original image fused with its corresponding true and partial diagonal images. To extract features, we adopted the generalized two-dimensional FLD (G2DFLD) feature extraction technique. The feature vectors from a test image are given as input to neural network-based classifier. It is trained with the feature vectors of original image and diagonally fused images and thereby the merit weights with respect to different classes were generated. To address the factors that affect the face recognition accuracy and uncertainty related to raw biometric data, a fuzzy score for each of the classes is generated by treating a type-2 fuzzy set. This type-2 fuzzy set is formed by the feature vectors of both the diagonally fused training samples and the test image of the respective classes. A concluding score for each of the classes under consideration is computed by fusing complemented merit weight with the complemented fuzzy score. These class-wise concluding scores are considered in the face recognition process. In this study, the well-known face databases (AT&T, UMIST and CMU-PIE) are used to evaluate the performance of the proposed method. The experimental results illustrate the fact that the proposed method has exhibited superior classification precision as compared with other state-of-art methods. Our T2FMFImgF method achieves highest face recognition accuracies of 99.41%, 98.36% and 89.80% in case of AT&T, UMIST and CMU-PIE (with expression), respectively while for CMU-PIE (with Light) the highest recognition accuracy is 97.957% . In addition to it, the presented method is quite successful in fusing and classifying textural information from the original and partial diagonal images by integrating them with type-2 fuzzy set-based treatment.