Spacetime Stereo and 3D Flow via Binocular Spatiotemporal Orientation Analysis

Forest 3D Reconstruction and Individual Tree Parameter Extraction Combining Close-Range Photo Enhancement and Feature Matching

An efficient and accurate forest sample plot survey is of great significance to understand the current status of forest resources at the stand or regional scale and the basis of scientific forest management. Close-range photogrammetry (CRP) technology can easily and quickly collect sequence images with high overlapping to reconstruct the 3D model of forest scenes and extract the individual tree parameters automatically and, therefore, can greatly improve the efficiency of forest investigation and has great application potential in forestry visualization management. However, it has some issues in practical forestry applications. First, the imaging quality is affected by the illumination in the forest, resulting in difficulty in feature matching and low accuracy of parameter extraction. Second, the efficiency of 3D forest model reconstruction is limited under complex understory vegetation or the topographic situation in the forest. In addition, the density of point clouds by dense matching directly affects the accuracy of individual tree parameter extraction. This research collected the sequence images of sample plots of four tree species by smartphones in Gaofeng Forest Farm in Guangxi and Wangyedian Forest Farm in Mongolia to analyze the effects of image enhancement, feature detection and dense point cloud algorithms on the efficiency of 3D forest reconstruction and accuracy of individual tree parameter extraction, then proposed a strategy of 3D reconstruction and parameter extraction suitable for different forest scenes. First, we compared the image enhancement effects of median–Gaussian (MG) filtering, single-scale retinex (SSR) and multi-scale retinex (MSR) filtering algorithms. Then, an improved algorithm combining Harris corner detection with speeded-up robust features (SURF) feature detection (Harris+SURF) is proposed, and the feature matching effect is compared with that of a scale invariant feature transform (SIFT) operator. Third, according to the morphological characteristics of the trees in the sequence images, we used the iterative interpolation algorithm of a planar triangulation network based on geometric constraints (GC-based IIPTN) to increase the density of point clouds and reconstruct the 3D forest model, and then extract the position and DBH of the individual trees. The results show that MSR image enhancement can significantly increase the number of matched point pairs. The improved Harris+SURF method can reduce the reconstruction time of the 3D forest model, and the GC-based IIPTN algorithm can improve the accuracy of individual tree parameter extraction. The extracted position of the individual tree is the same as the measured position with the bias within 0.2 m. The accuracy of extracted DBH of Eucalyptus grandis, Taxus chinensis, Larix gmelinii and Pinus tabuliformis is 94%, 95%, 96% and 90%, respectively, which proves that the proposed 3D model reconstruction method based on image enhancement has great potential for tree position and DBH extraction, and also provides effective support for forest resource investigation and visualization management in the future.

Dynamic Scene Recognition with Complementary Spatiotemporal Features

This paper presents Dynamically Pooled Complementary Features (DPCF), a unified approach to dynamic scene recognition that analyzes a short video clip in terms of its spatial, temporal and color properties. The complementarity of these properties is preserved through all main steps of processing, including primitive feature extraction, coding and pooling. In the feature extraction step, spatial orientations capture static appearance, spatiotemporal oriented energies capture image dynamics and color statistics capture chromatic information. Subsequently, primitive features are encoded into a mid-level representation that has been learned for the task of dynamic scene recognition. Finally, a novel dynamic spacetime pyramid is introduced. This dynamic pooling approach can handle both global as well as local motion by adapting to the temporal structure, as guided by pooling energies. The resulting system provides online recognition of dynamic scenes that is thoroughly evaluated on the two current benchmark datasets and yields best results to date on both datasets. In-depth analysis reveals the benefits of explicitly modeling feature complementarity in combination with the dynamic spacetime pyramid, indicating that this unified approach should be well-suited to many areas of video analysis.

Quality Index for Stereoscopic Images by Separately Evaluating Adding and Subtracting

The human visual system (HVS) plays an important role in stereo image quality perception. Therefore, it has aroused many people’s interest in how to take advantage of the knowledge of the visual perception in image quality assessment models. This paper proposes a full-reference metric for quality assessment of stereoscopic images based on the binocular difference channel and binocular summation channel. For a stereo pair, the binocular summation map and binocular difference map are computed first by adding and subtracting the left image and right image. Then the binocular summation is decoupled into two parts, namely additive impairments and detail losses. The quality of binocular summation is obtained as the adaptive combination of the quality of detail losses and additive impairments. The quality of binocular summation is computed by using the Contrast Sensitivity Function (CSF) and weighted multi-scale (MS-SSIM). Finally, the quality of binocular summation and binocular difference is integrated into an overall quality index. The experimental results indicate that compared with existing metrics, the proposed metric is highly consistent with the subjective quality assessment and is a robust measure. The result have also indirectly proved hypothesis of the existence of binocular summation and binocular difference channels.