Portable System for Box Volume Measurement Based on Line-Structured Light Vision and Deep Learning

Portable box volume measurement has always been a popular issue in the intelligent logistic industry. This work presents a portable system for box volume measurement that is based on line-structured light vision and deep learning. This system consists of a novel 2 × 2 laser line grid projector, a sensor, and software modules, with which only two laser-modulated images of boxes are required for volume measurement. For laser-modulated images, a novel end-to-end deep learning model is proposed by using an improved holistically nested edge detection network to extract edges. Furthermore, an automatic one-step calibration method for the line-structured light projector is designed for fast calibration. The experimental results show that the measuring range of our proposed system is 100–1800 mm, with errors less than ±5.0 mm. Theoretical analysis indicates that within the measuring range of the system, the measurement uncertainty of the measuring device is ±0.52 mm to ±4.0 mm, which is consistent with the experimental results. The device size is 140 mm × 35 mm × 35 mm and the weight is 110 g, thus the system is suitable for portable automatic box volume measurement.

Self-recalibration of a robot-assisted structured-light-based measurement system

The structured-light-based measurement method is widely employed in numerous fields. However, for industrial inspection, to achieve complete scanning of a work piece and overcome occlusion, the measurement system needs to be moved to different viewpoints. Moreover, frequent reconfiguration of the measurement system may be needed based on the size of the measured object, making the self-recalibration of extrinsic parameters indispensable. To this end, this paper proposes an automatic self-recalibration and reconstruction method, wherein a robot arm is employed to move the measurement system for complete scanning; the self-recalibration is achieved using fundamental matrix calculations and point cloud registration without the need for an accurate calibration gauge. Experimental results demonstrate the feasibility and accuracy of our method.

Robust active stereo vision using Kullback-Leibler divergence

Active stereo vision is a method of 3D surface scanning involving the projecting and capturing of a series of light patterns where depth is derived from correspondences between the observed and projected patterns. In contrast, passive stereo vision reveals depth through correspondences between textured images from two or more cameras. By employing a projector, active stereo vision systems find correspondences between two or more cameras, without ambiguity, independent of object texture. In this paper, we present a hybrid 3D reconstruction framework that supplements projected pattern correspondence matching with texture information. The proposed scheme consists of using projected pattern data to derive initial correspondences across cameras and then using texture data to eliminate ambiguities. Pattern modulation data are then used to estimate error models from which Kullback-Leibler divergence refinement is applied to reduce misregistration errors. Using only a small number of patterns, the presented approach reduces measurement errors versus traditional structured light and phase matching methodologies while being insensitive to gamma distortion, projector flickering, and secondary reflections. Experimental results demonstrate these advantages in terms of enhanced 3D reconstruction performance in the presence of noise, deterministic distortions, and conditions of texture and depth contrast.

CODED STRUCTURED LIGHT STRIPE BOUNDARY LOCATION WITH SUB-PIXEL ACCURACY BASED ON WEIGHTED CENTROID METHOD

Sub-pixel accuracy stripe boundary location is essential for stripe coded structured light based 3D reconstruction. So far, the intersecting point method might be the most accurate for stripe boundary location. But in this method two groups of patterns have to be projected, where one group is normal and the other is inverse patterns. So the intersecting point method can not be used in several time- or resource-constrained applications because of its complexity of projection. In this paper, after a brief survey on several commonly used stripe boundary detection methods, an improved sub-pixel location scheme based on weighted centroid algorithm is proposed. In the proposed scheme, a green slit is first added between every stripe to avoid false detection and false rejection. Then, universal gravity-based edge detection and weighted centroid method are utilized for sub-pixel location. Compared with the intersecting point method, the proposed location scheme can achieve comparable detection accuracy with only half number of patterns projected.

Period coded phase shifting strategy for real-time 3-D structured light illumination

Phase shifting structured light illumination for range sensing involves projecting a set of grating patterns where accuracy is determined, in part, by the number of stripes. However, high pattern frequencies introduce ambiguities during phase unwrapping. This paper proposes a process for embedding a period cue into the projected pattern set without reducing the signal-to-noise ratio. As a result, each period of the high frequency signal can be identified. The proposed method can unwrap high frequency phase and achieve high measurement precision without increasing the pattern number. Therefore, the proposed method can significantly benefit real-time applications. The method is verified by theoretical and experimental analysis using prototype system built to achieve 120 fps at 640 × 480 resolution.

Mobile calibration based on laser metrology and approximation networks

A mobile calibration technique for three-dimensional vision is presented. In this method, vision parameters are computed automatically by approximation networks built based on the position of a camera and image processing of a laser line. The networks also perform three-dimensional visualization. In the proposed system, the setup geometry can be modified online, whereby an online re-calibration is performed based on data provided by the network and the required modifications of extrinsic and intrinsic parameters are thus determined, overcoming any calibration limitations caused by the modification procedure. The mobile calibration also avoids procedures involving references, which are used in traditional online re-calibration methods. The proposed mobile calibration thus improves the accuracy and performance of the three-dimensional vision because online data of calibrated references are not passed on to the vision system. This work represents a contribution to the field of online re-calibration, as verified by a comparison with the results based on lighting methods, which are calibrated and re-calibrated via perspective projection. Processing time is also studied.

Multiresolution approach based on projection matrices

Active triangulation measurement systems with a rigid geometric configuration are inappropriate for scanning large objects with low measuring tolerances. The reason is that the ratio between the depth recovery error and the lateral extension is a constant that depends on the geometric setup. As a consequence, measuring large areas with low depth recovery error requires the use of multiresolution techniques. We propose a multiresolution technique based on a camera-projector system previously calibrated. The method consists of changing the camera or projector's parameters in order to increase the system depth sensitivity. A subpixel retroprojection error in the self-calibration process and a decrease of approximately one order of magnitude in the depth recovery error can be achieved using the proposed method.

Vision processing for realtime 3-D data acquisition based on coded structured light

Structured light vision systems have been successfully used for accurate measurement of 3-D surfaces in computer vision. However, their applications are mainly limited to scanning stationary objects so far since tens of images have to be captured for recovering one 3-D scene. This paper presents an idea for real-time acquisition of 3-D surface data by a specially coded vision system. To achieve 3-D measurement for a dynamic scene, the data acquisition must be performed with only a single image. A principle of uniquely color-encoded pattern projection is proposed to design a color matrix for improving the reconstruction efficiency. The matrix is produced by a special code sequence and a number of state transitions. A color projector is controlled by a computer to generate the desired color patterns in the scene. The unique indexing of the light codes is crucial here for color projection since it is essential that each light grid be uniquely identified by incorporating local neighborhoods so that 3-D reconstruction can be performed with only local analysis of a single image. A scheme is presented to describe such a vision processing method for fast 3-D data acquisition. Practical experimental performance is provided to analyze the efficiency of the proposed methods.

Toward dynamic recalibration and three-dimensional reconstruction in a structured light system

We present a method for dynamic recalibration and 3D reconstruction via a structured light system. Assuming that the light planes cast from the digital light projector have been calibrated off-line, we show that the focal length, aspect ratio, and all motion parameters of the camera can be determined on-line. Then the 3D reconstruction can be carried out by either a traditional triangulation method or a more efficient transformation-based method. In the latter method, a single image is sufficient for the whole process of calibration and reconstruction. Thus a hand-held camera can be used. Computer simulation and real data experiments were carried out to validate the method.

Vision Sensor Planning for 3–D Model Acquisition

A novel method is proposed in this paper for automatic acquisition of three-dimensional (3-D) models of unknown objects by an active vision system, in which the vision sensor is to be moved from one viewpoint to the next around the target to obtain its complete model. In each step, sensing parameters are determined automatically for incrementally building the 3-D target models. The method is developed by analyzing the target's trend surface, which is the regional feature of a surface for describing the global tendency of change. While previous approaches to trend analysis are usually focused on generating polynomial equations for interpreting regression surfaces in three dimensions, this paper proposes a new mathematical model for predicting the unknown area of the object surface. A uniform surface model is established by analyzing the surface curvatures. Furthermore, a criterion is defined to determine the exploration direction, and an algorithm is developed for determining the parameters of the next view. Implementation of the method is carried out to validate the proposed method.