Neural-Network-Based Model-Free Calibration Method for Stereo Fisheye Camera

Research on calibration of a binocular stereo-vision imaging system based on the artificial neural network

Camera calibration is a key problem for 3D reconstruction in computer vision. Existing calibration methods, such as traditional, active, and self-calibration, all need to solve the internal and external parameters of the imaging system to clarify the image-object mapping relationship. The artificial neural network, which is based on connectionist architecture, provides a novel idea for the calibration of nonlinear mapping vision systems. It can learn the image-object mapping relationship from some sample points without considering too many uncertain factors in the middle. This paper discusses the learning ability. A binocular stereo-vision mapping model is used as the learning model to explore the ability of image-object mapping for artificial neural networks. This paper constructs sample libraries by pixel and world coordinates of checkerboard corners, builds the artificial neural network, and, through the training samples and test samples prediction, verifies the learning performance of the network. Furthermore, by the laser scanning binocular vision device constructed in the authors' laboratory and trained-well network, the 3D point cloud reconstruction of a physical target is performed. The experimental results show that the artificial neural network can learn the image-object mapping relationship well and more effectively avoid the impact of lens distortion and achieve more accurate nonlinear mapping at the edge of the image. When the X and Y coordinates are in the range of 100 mm and the Z coordinates are in the range of a 1000 mm, the absolute error rarely exceeds 2.5 mm, and the relative error is in the level of 10^-3; for 1000 mm distance measurement, the standard deviation does not exceed 1.5 mm. Network parameter selection experiments show that, for image-object mapping, a three-layer network and increasing the number of hidden layer's nodes can improve the training time more significantly.

Distributed Network of Adaptive and Self-Reconfigurable Active Vision Systems

The performance of a computer vision system depends on the accuracy of visual information extracted by the sensors and the system’s visual-processing capabilities. To derive optimum information from the sensed data, the system must be capable of identifying objects of interest (OOIs) and activities in the scene. Active vision systems intend to capture OOIs with the highest possible resolution to extract the optimum visual information by calibrating the configuration spaces of the cameras. As the data processing and reconfiguration of cameras are interdependent, it becomes very challenging for advanced active vision systems to perform in real time. Due to limited computational resources, model-based asymmetric active vision systems only work in known conditions and fail miserably in unforeseen conditions. Symmetric/asymmetric systems employing artificial intelligence, while they manage to tackle unforeseen environments, require iterative training and thus are not reliable for real-time applications. Thus, the contemporary symmetric/asymmetric reconfiguration systems proposed to obtain optimum configuration spaces of sensors for accurate activity tracking and scene understanding may not be adequate to tackle unforeseen conditions in real time. To address this problem, this article presents an adaptive self-reconfiguration (ASR) framework for active vision systems operating co-operatively in a distributed blockchain network. The ASR framework enables active vision systems to share their derived learning about an activity or an unforeseen environment, which learning can be utilized by other active vision systems in the network, thus lowering the time needed for learning and adaptation to new conditions. Further, as the learning duration is reduced, the duration of the reconfiguration of the cameras is also reduced, yielding better performance in terms of understanding of a scene. The ASR framework enables resource and data sharing in a distributed network of active vision systems and outperforms state-of-the-art active vision systems in terms of accuracy and latency, making it ideal for real-time applications.