Blind assessment for stereo images considering binocular characteristics and deep perception map based on deep belief network

Fast reconstruction method of three-dimension model based on dual RGB-D cameras for peanut plant

BACKGROUND

Plant shape and structure are important factors in peanut breeding research. Constructing a three-dimension (3D) model can provide an effective digital tool for comprehensive and quantitative analysis of peanut plant structure. Fast and accurate are always the goals of the plant 3D model reconstruction research.

RESULTS

We proposed a 3D reconstruction method based on dual RGB-D cameras for the peanut plant 3D model quickly and accurately. The two Kinect v2 were mirror symmetry placed on both sides of the peanut plant, and the point cloud data obtained were filtered twice to remove noise interference. After rotation and translation based on the corresponding geometric relationship, the point cloud acquired by the two Kinect v2 was converted to the same coordinate system and spliced into the 3D structure of the peanut plant. The experiment was conducted at various growth stages based on twenty potted peanuts. The plant traits' height, width, length, and volume were calculated through the reconstructed 3D models, and manual measurement was also carried out during the experiment processing. The accuracy of the 3D model was evaluated through a synthetic coefficient, which was generated by calculating the average accuracy of the four traits. The test result showed that the average accuracy of the reconstructed peanut plant 3D model by this method is 93.42%. A comparative experiment with the iterative closest point (ICP) algorithm, a widely used 3D modeling algorithm, was additionally implemented to test the rapidity of this method. The test result shows that the proposed method is 2.54 times faster with approximated accuracy compared to the ICP method.

CONCLUSIONS

The reconstruction method for the 3D model of the peanut plant described in this paper is capable of rapidly and accurately establishing a 3D model of the peanut plant while also meeting the modeling requirements for other species' breeding processes. This study offers a potential tool to further explore the 3D model for improving traits and agronomic qualities of plants.

HFFNet: hierarchical feature fusion network for blind binocular image quality prediction

Compared with monocular images, scene discrepancies between the left- and right-view images impose additional challenges on visual quality predictions in binocular images. Herein, we propose a hierarchical feature fusion network (HFFNet) for blind binocular image quality prediction that handles scene discrepancies and uses multilevel fusion features from the left- and right-view images to reflect distortions in binocular images. Specifically, a feature extraction network based on MobileNetV2 is used to determine the feature layers from distorted binocular images; then, low-level binocular fusion features (or middle-level and high-level binocular fusion features) are obtained by fusing the left and right low-level monocular features (or middle-level and high-level monocular features) using the feature gate module; further, three feature enhancement modules are used to enrich the information of the extracted features at different levels. Finally, the total feature maps obtained from the high-, middle-, and low-level fusion features are applied to a three-input feature fusion module for feature merging. Thus, the proposed HFFNet provides better results, to the best of our knowledge, than existing methods on two benchmark datasets.

A no-Reference Stereoscopic Image Quality Assessment Network Based on Binocular Interaction and Fusion Mechanisms

In contemporary society full of stereoscopic images, how to assess visual quality of 3D images has attracted an increasing attention in field of Stereoscopic Image Quality Assessment (SIQA). Compared with 2D-IQA, SIQA is more challenging because some complicated features of Human Visual System (HVS), such as binocular interaction and binocular fusion, must be considered. In this paper, considering both binocular interaction and fusion mechanisms of the HVS, a hierarchical no-reference stereoscopic image quality assessment network (StereoIF-Net) is proposed to simulate the whole quality perception of 3D visual signals in human cortex, including two key modules: BIM and BFM. In particular, Binocular Interaction Modules (BIMs) are constructed to simulate binocular interaction in V2-V5 visual cortex regions, in which a novel cross convolution is designed to explore the interaction details in each region. In the BIMs, different output channel numbers are designed to imitate various receptive fields in V2-V5. Furthermore, a Binocular Fusion Module (BFM) with automatic learned weights is proposed to model binocular fusion of the HVS in higher cortex layers. The verification experiments are conducted on the LIVE 3D, IVC and Waterloo-IVC SIQA databases and three indices including PLCC, SROCC and RMSE are employed to evaluate the assessment consistency between StereoIF-Net and the HVS. The proposed StereoIF-Net achieves almost the best results compared with advanced SIQA methods. Specifically, the metric values on LIVE 3D, IVC and WIVC-I are the best, and are the second-best on the WIVC-II.

3D Virtual Reality Implementation of Tourist Attractions Based on the Deep Belief Neural Network

In today's society, information technology is widely used, and virtual reality technology, as one of the emerging frontier technologies, has entered a stage of rapid development. Virtual reality is the use of computer technology to simulate the real-life environment into a virtual simulation environment, with the help of special equipment to realize the natural interaction between users and technical environment, in which the tourism industry is the most widely used. In order to realize 3D virtual reality of tourist attractions and improve users' immersive experience in the process of interaction, the deep belief neural network is introduced to realize the target recognition and reconstruction in virtual reality. The results show that the algorithm has excellent performance in target recognition and target reconstruction, and deep belief networks improve the accuracy by 0.57% and 0.81% and the accuracy by 0.21% and 2.06%, respectively, compared with the current optimal algorithm in target recognition of 12 and 20 view regular projection images. Compared with the current optimal algorithm, deep belief networks are reduced by 0.2%, 3.7%, and 0.6%, respectively. The accuracy index was increased by 2%, 0.1%, and 0.1%, respectively. The above results show that the proposed algorithm based on the deep belief neural network can realize 3D virtual reality of complex scenes such as tourist attractions according to its excellent performance.

Autonomous Operation Method of Multi-DOF Robotic Arm Based on Binocular Vision

Robotic arms with autonomous operation capabilities are currently widely used in real life, such as fruit picking, cargo handling, and workpiece assembly. However, the common autonomous operation methods of the robotic arm have some disadvantages, such as poor universality, low robustness, and difficult implementation. An autonomous operation method of multi-DOF (Multiple Degree of Freedom) robotic arm is proposed in this study on the basis of binocular vision. First, a target object extraction method based on extracting target feature points is proposed, and combined with the binocular positioning principle to obtain the spatial position of the target object. Second, in order to improve the working efficiency of the robotic arm, the robotic arm motion trajectory is planned on the basis of genetic algorithm in the joint space. Finally, a small physical prototype model is built for experimental verification. The experimental results show that the relative positioning error of the target object can reach 1.19% in the depth of field of 70–200 mm. The average grab error, variance, and grab success rate of the robot arm are 14 mm, 6.5 mm, and 83%, respectively. This shows that the method proposed in this paper has the advantages of high robustness, good versatility and easy implementation.

Dual-Stream Interactive Networks for No-Reference Stereoscopic Image Quality Assessment

The goal of objective stereoscopic image quality assessment (SIQA) is to predict the human perceptual quality of stereoscopic/3D images automatically and accurately. Compared with traditional 2D image quality assessment, the quality assessment of stereoscopic images is more challenging because of complex binocular vision mechanisms and multiple quality dimensions. In this paper, inspired by the hierarchical dual-stream interactive nature of the human visual system, we propose a stereoscopic image quality assessment network (StereoQA-Net) for no-reference stereoscopic image quality assessment. The proposed StereoQA-Net is an end-to-end dual-stream interactive network containing left and right view sub-networks, where the interaction of the two sub-networks exists in multiple layers. We evaluate our method on the LIVE stereoscopic image quality databases. The experimental results show that our proposed StereoQA-Net outperforms state-of-the-art algorithms on both symmetrically and asymmetrically distorted stereoscopic image pairs of various distortion types. In a more general case, the proposed StereoQA-Net can effectively predict the perceptual quality of local regions. In addition, cross-dataset experiments also demonstrate the generalization ability of our algorithm.