Comparison of 3D Sensors for Automating Bolt-Tightening Operations in the Automotive Industry

Machine vision systems are widely used in assembly lines for providing sensing abilities to robots to allow them to handle dynamic environments. This paper presents a comparison of 3D sensors for evaluating which one is best suited for usage in a machine vision system for robotic fastening operations within an automotive assembly line. The perception system is necessary for taking into account the position uncertainty that arises from the vehicles being transported in an aerial conveyor. Three sensors with different working principles were compared, namely laser triangulation (SICK TriSpector1030), structured light with sequential stripe patterns (Photoneo PhoXi S) and structured light with infrared speckle pattern (Asus Xtion Pro Live). The accuracy of the sensors was measured by computing the root mean square error (RMSE) of the point cloud registrations between their scans and two types of reference point clouds, namely, CAD files and 3D sensor scans. Overall, the RMSE was lower when using sensor scans, with the SICK TriSpector1030 achieving the best results (0.25 mm ± 0.03 mm), the Photoneo PhoXi S having the intermediate performance (0.49 mm ± 0.14 mm) and the Asus Xtion Pro Live obtaining the higher RMSE (1.01 mm ± 0.11 mm). Considering the use case requirements, the final machine vision system relied on the SICK TriSpector1030 sensor and was integrated with a collaborative robot, which was successfully deployed in an vehicle assembly line, achieving 94% success in 53,400 screwing operations.

NIR-Triggered High-Efficiency Self-Healable Protective Optical Coating for Vision Systems

Recently, self-healing materials have evolved to recover specific functions such as electronic, magnetic, acoustic, structural or hierarchical, and biological properties. In particular, the development of self-healing protection coatings that can be applied to lens components in vision systems such as augmented reality glasses, actuators, and image and time-of-flight sensors has received intensive attention from the industry. In the present study, we designed polythiourethane dynamic networks containing a photothermal N-butyl-substituted diimmonium borate dye to demonstrate their potential applications in self-healing protection coatings for the optical components of vision systems. The optimized self-healing coating exhibited a high transmittance (∼95% in the visible-light region), tunable refractive index (up to 1.6), a moderate Abbe number (∼35), and high surface hardness (>200 MPa). When subjected to near-infrared (NIR) radiation (1064 nm), the surface temperature of the coating increased to 75 °C via the photothermal effect and self-healing of the scratched coatings occurred via a dynamic thiourethane exchange reaction. The coating was applied to a lens protector, and its self-healing performance was demonstrated. The light signal distorted by the scratched surface of the coating was perfectly restored after NIR-induced self-healing. The photoinduced self-healing process can also autonomously occur under sunlight with low energy consumption.

Information Fusion of GPS, INS and Odometer Sensors for Improving Localization Accuracy of Mobile Robots in Indoor and Outdoor Applications

In mobile robot localization with multiple sensors, myriad problems arise as a result of inadequacies associated with each of the individual sensors. In such cases, methodologies built upon the concept of multisensor fusion are well-known to provide optimal solutions and overcome issues such as sensor nonlinearities and uncertainties. Artificial neural networks and fuzzy logic (FL) approaches can effectively model sensors with unknown nonlinearities and uncertainties. In this article, a robust approach for localization (positioning) of a mobile robot in indoor as well as outdoor environments is proposed. The neural network is utilized as a pseudo-sensor that models the global positioning system (GPS) and is used to predict the robot’s position in case of GPS signal loss in indoor environments. The data from proprioceptive sensors such as inertial sensors and GPS are fused using the Kalman and the complementary filter-based fusion schemes in the outdoor case. To eliminate the position inaccuracies due to wheel slippage, an expert FL system (FLS) is implemented and cascaded with the sensor fusion module. The proposed technique is tested both in simulation and in real scenarios of robot movements. The simulations and results from the experimental platform validate the efficacy of the proposed algorithm.

Fast Depth Estimation in a Single Image Using Lightweight Efficient Neural Network

Depth estimation is a crucial and fundamental problem in the computer vision field. Conventional methods re-construct scenes using feature points extracted from multiple images; however, these approaches require multiple images and thus are not easily implemented in various real-time applications. Moreover, the special equipment required by hardware-based approaches using 3D sensors is expensive. Therefore, software-based methods for estimating depth from a single image using machine learning or deep learning are emerging as new alternatives. In this paper, we propose an algorithm that generates a depth map in real time using a single image and an optimized lightweight efficient neural network (L-ENet) algorithm instead of physical equipment, such as an infrared sensor or multi-view camera. Because depth values have a continuous nature and can produce locally ambiguous results, pixel-wise prediction with ordinal depth range classification was applied in this study. In addition, in our method various convolution techniques are applied to extract a dense feature map, and the number of parameters is greatly reduced by reducing the network layer. By using the proposed L-ENet algorithm, an accurate depth map can be generated from a single image quickly and, in a comparison with the ground truth, we can produce depth values closer to those of the ground truth with small errors. Experiments confirmed that the proposed L-ENet can achieve a significantly improved estimation performance over the state-of-the-art algorithms in depth estimation based on a single image.