Research on computer vision enhancement in intelligent robot based on machine learning and deep learning

Research on Multi-Hole Localization Tracking Based on a Combination of Machine Vision and Deep Learning

In the process of industrial production, manual assembly of workpieces exists with low efficiency and high intensity, and some of the assembly process of the human body has a certain degree of danger. At the same time, traditional machine learning algorithms are difficult to adapt to the complexity of the current industrial field environment; the change in the environment will greatly affect the accuracy of the robot's work. Therefore, this paper proposes a method based on the combination of machine vision and the YOLOv5 deep learning model to obtain the disk porous localization information, after coordinate mapping by the ROS communication control robotic arm work, in order to improve the anti-interference ability of the environment and work efficiency but also reduce the danger to the human body. The system utilizes a camera to collect real-time images of targets in complex environments and, then, trains and processes them for recognition such that coordinate localization information can be obtained. This information is converted into coordinates under the robot coordinate system through hand-eye calibration, and the robot is then controlled to complete multi-hole localization and tracking by means of communication between the upper and lower computers. The results show that there is a high accuracy in the training and testing of the target object, and the control accuracy of the robotic arm is also relatively high. The method has strong anti-interference to the complex environment of industry and exhibits a certain feasibility and effectiveness. It lays a foundation for achieving the automated installation of docking disk workpieces in industrial production and also provides a more favorable choice for the production and installation of the process of screw positioning needs.

A novel multidimensional uncalibration method applied to six-axis manipulators

This study proposes a multidimensional uncalibrated technique for tracking and grasping dynamic targets by a robotic arm in the eye-in-hand mode. This method avoids complex and cumbersome calibration processes, enabling machine vision tasks to be adaptively applied in a variety of complex environments, which solved the problem of traditional calibration methods being unstable in complex environments. The specific method used in this study is first, in the eye-in-hand mode, the robotic arm moves along the x, y, and z axes in sequence, and images are taken before and after each movement. Thereafter, the image Jacobian matrix is calculated from the three (or more) sets of images collected. Finally, the robotic arm converts the target coordinates in the real-time captured images by the camera into coordinates in the robotic arm coordinate system through the image Jacobian matrix and performs real-time tracking. This study tests the dynamic quasi-Newton method for estimating the Jacobian matrix and optimizes the initialization coupling problem using the orthogonal moving method. This optimization scheme significantly shortens the iteration process, making the uncalibrated technology more fully applied in the field of dynamic object tracking. In addition, this study proposes a servo control algorithm with predictive compensation to mitigate or even eliminate the systematic error caused by time delay in dynamic target tracking in robot visual servo systems.

Fast and Accurate Pose Estimation with Unknown Focal Length Using Line Correspondences

Estimating camera pose is one of the key steps in computer vison, photogrammetry and SLAM (Simultaneous Localization and Mapping). It is mainly calculated based on the 2D-3D correspondences of features, including 2D-3D point and line correspondences. If a zoom lens is equipped, the focal length needs to be estimated simultaneously. In this paper, a new method of fast and accurate pose estimation with unknown focal length using two 2D-3D line correspondences and the camera position is proposed. Our core contribution is to convert the PnL (perspective-n-line) problem with 2D-3D line correspondences into an estimation problem with 3D-3D point correspondences. One 3D line and the camera position in the world frame can define a plane, the 2D line projection of the 3D line and the camera position in the camera frame can define another plane, and actually the two planes are the same plane, which is the key geometric characteristic in this paper's estimation of focal length and pose. We establish the transform between the normal vectors of the two planes with this characteristic, and this transform can be regarded as the camera projection of a 3D point. Then, the pose estimation using 2D-3D line correspondences is converted into pose estimation using 3D-3D point correspondences in intermediate frames, and, lastly, pose estimation can be finished quickly. In addition, using the property whereby the angle between two planes is invariant in both the camera frame and world frame, we can estimate the camera focal length quickly and accurately. Experimental results show that our proposed method has good performance in numerical stability, noise sensitivity and computational speed with synthetic data and real scenarios, and has strong robustness to camera position noise.

Distance Assessment by Object Detection—For Visually Impaired Assistive Mechatronic System

Techniques for the detection and recognition of objects have experienced continuous development over recent years, as their application and benefits are so very obvious. Whether they are involved in driving a car, environment surveillance and security, or assistive living for people with different disabilities, not to mention advanced robotic surgery, these techniques are almost indispensable. This article presents the research results of a distance assessment using object detection and recognition techniques. The first is a new technique based on low-cost photo cameras and special sign detection. The second is a classic approach based on a LIDAR sensor and an HQ photo camera. Its novelty, in this case, consists of the concept and prototype of the hardware subsystem for high-precision distance measurement, as well as fast and accurate object recognition. The experimentally obtained results are used for the motion control strategy (directional inverse kinematics) of the robotic arm (virtual prototype) component in special assistive devices designed for visually impaired persons. The advantages of the original technical solution, experimentally validated by a prototype system with modern equipment, are the precision and the short time required for the identification and recognition of objects at relatively short distances. The research results obtained, in both the real and virtual experiments, stand as a basis for the further development of the visually impaired mechatronic system prototype using additional ultrasonic sensors, stereoscopic or multiple cameras, and the implementation of machine-learning models for safety-critical tasks.

Air-Quality Prediction Based on the EMD–IPSO–LSTM Combination Model

Owing to climate change, industrial pollution, and population gathering, the air quality status in many places in China is not optimal. The continuous deterioration of air-quality conditions has considerably affected the economic development and health of China’s people. However, the diversity and complexity of the factors which affect air pollution render air quality monitoring data complex and nonlinear. To improve the accuracy of prediction of the air quality index (AQI) and obtain more accurate AQI data with respect to their nonlinear and nonsmooth characteristics, this study introduces an air quality prediction model based on the empirical mode decomposition (EMD) of LSTM and uses improved particle swarm optimization (IPSO) to identify the optimal LSTM parameters. First, the model performed the EMD decomposition of air quality data and obtained uncoupled intrinsic mode function (IMF) components after removing noisy data. Second, we built an EMD–IPSO–LSTM air quality prediction model for each IMF component and extracted prediction values. Third, the results of validation analyses of the algorithm showed that compared with LSTM and EMD–LSTM, the improved model had higher prediction accuracy and improved the model fitting effect, which provided theoretical and technical support for the prediction and management of air pollution.

A Survey of Multi-Agent Cross Domain Cooperative Perception

Intelligent unmanned systems for ground, sea, aviation, and aerospace application are important research directions for the new generation of artificial intelligence in China. Intelligent unmanned systems are also important carriers of interactive mapping between physical space and cyberspace in the process of the digitization of human society. Based on the current domestic and overseas development status of unmanned systems for ground, sea, aviation, and aerospace application, this paper reviewed the theoretical problems and research trends of multi-agent cross-domain cooperative perception. The scenarios of multi-agent cooperative perception tasks in different areas were deeply investigated and analyzed, the scientific problems of cooperative perception were analyzed, and the development direction of multi-agent cooperative perception theory research for solving the challenges of the complex environment, interactive communication, and cross-domain tasks was expounded.