Gimbal Influence on the Stability of Exterior Orientation Parameters of UAV Acquired Images

An LADRC Controller to Improve the Robustness of the Visual Tracking and Inertial Stabilized System in Luminance Variation Conditions

Disturbance from luminance variation in the identification of visual sensors causes instability in the control system of target tracking, which leads to field of vision (FOV) motion and even the target missing. To solve this problem, a linear active disturbance reject controller (LADRC) is adopted to the visual tracking and inertial stable platform (VTISP) for the first time to improve the system’s robustness. As a result, the random disturbance from identification can be smoothed by the tracking differentiator (TD).An improved linear extended state observer (LESO) modified by the TD is provided to obtain the high-order state variables for feedback. That makes the system avoid noise in a differential process from the MEMS gyroscope and enhances the response time and stability in tracking control. Finally, simulation and experimental studies are conducted, and the feasibility of the LADRC is verified. Moreover, compared with the other controller in the VTISP for remote sensing, the superiority of the LADRC in system response time and stability is proved by the experiments.

A Hierarchical Stabilization Control Method for a Three-Axis Gimbal Based on Sea-Sky-Line Detection

Obtaining a stable video sequence for cameras on surface vehicles is always a challenging problem due to the severe disturbances in heavy sea environments. Aiming at this problem, this paper proposes a novel hierarchical stabilization method based on real-time sea–sky-line detection. More specifically, a hierarchical image stabilization control method that combines mechanical image stabilization with electronic image stabilization is adopted. With respect to the mechanical image stabilization method, a gimbal with three degrees of freedom (DOFs) and with a robust controller is utilized for the primary motion compensation. In addition, the electronic image stabilization method based on sea–sky-line detection in video sequences accomplishes motion estimation and compensation. The Canny algorithm and Hough transform are utilized to detect the sea–sky line. Noticeably, an image-clipping strategy based on prior information is implemented to ensure real-time performance, which can effectively improve the processing speed and reduce the equipment performance requirements. The experimental results indicate that the proposed method for mechanical and electronic stabilization can reduce the vibration by 74.2% and 42.1%, respectively.

Quantifying Marine Macro Litter Abundance on a Sandy Beach Using Unmanned Aerial Systems and Object-Oriented Machine Learning Methods

Unmanned aerial systems (UASs) have recently been proven to be valuable remote sensing tools for detecting marine macro litter (MML), with the potential of supporting pollution monitoring programs on coasts. Very low altitude images, acquired with a low-cost RGB camera onboard a UAS on a sandy beach, were used to characterize the abundance of stranded macro litter. We developed an object-oriented classification strategy for automatically identifying the marine macro litter items on a UAS-based orthomosaic. A comparison is presented among three automated object-oriented machine learning (OOML) techniques, namely random forest (RF), support vector machine (SVM), and k-nearest neighbor (KNN). Overall, the detection was satisfactory for the three techniques, with mean F-scores of 65% for KNN, 68% for SVM, and 72% for RF. A comparison with manual detection showed that the RF technique was the most accurate OOML macro litter detector, as it returned the best overall detection quality (F-score) with the lowest number of false positives. Because the number of tuning parameters varied among the three automated machine learning techniques and considering that the three generated abundance maps correlated similarly with the abundance map produced manually, the simplest KNN classifier was preferred to the more complex RF. This work contributes to advances in remote sensing marine litter surveys on coasts, optimizing the automated detection on UAS-derived orthomosaics. MML abundance maps, produced by UAS surveys, assist coastal managers and authorities through environmental pollution monitoring programs. In addition, they contribute to search and evaluation of the mitigation measures and improve clean-up operations on coastal environments.

Experimental Verification of Three-Degree-of-Freedom Electromagnetic Actuator for Image Stabilization

Image deteriorations due to vibrations have become a problem in autonomous systems such as unmanned aerial vehicles, robots, and autonomous cars. To suppress the vibration, a camera stabilizer using a gimbal mechanism is widely used. However, the size and weight of the system increase because the conventional image stabilization systems require some actuators and links to drive in multi-axes. In order to solve these problems, we proposed a novel three-degree-of-freedom (3DOF) electromagnetic actuator for image stabilization. The actuator can be driven by only three-phase and has a simple structure and control system. This paper describes the experimental verification of the proposed actuator. The torque characteristics are clarified, and the analysis and measured torque characteristics are compared to verify the analysis validity. For verifying the dynamic performance, the frequency characteristics are measured. The effectiveness of the proposed magnetic structure and operating principle are investigated.

Small Multicopter-UAV-Based Radar Imaging: Performance Assessment for a Single Flight Track

This paper deals with a feasibility study assessing the reconstruction capabilities of a small Multicopter-Unmanned Aerial Vehicle (M-UAV) based radar system, whose flight positions are determined by using the Carrier-Phase Differential GPS (CDGPS) technique. The paper describes the overall radar imaging system in terms of both hardware devices and data processing strategy for the case of a single flight track. The data processing is cast as the solution of an inverse scattering problem and is able to provide focused images of on surface targets. In particular, the reconstruction is approached through the adjoint of the functional operator linking the unknown contrast function to the scattered field data, which is computed by taking into account the actual flight positions provided by the CDGPS technique. For this inverse problem, we provide an analysis of the reconstruction capabilities by showing the effect of the radar parameters, the flight altitude and the spatial offset between target and flight path on the resolution limits. A measurement campaign is carried out to demonstrate the imaging capabilities in controlled conditions. Experimental results referred to two surveys performed on the same scene but at two different UAV altitudes verify the consistency of these results with the theoretical resolution analysis.

Design and Analysis for Early Warning of Rotor UAV Based on Data-Driven DBN

The unmanned aerial vehicle (UAV), which is a typical multi-sensor closed-loop flight control system, has the properties of multivariable, time-varying, strong coupling, and nonlinearity. Therefore, it is very difficult to obtain an accurate mathematical diagnostic model based on the traditional model-based method; this paper proposes a UAV sensor diagnostic method based on data-driven methods, which greatly improves the reliability of the rotor UAV nonlinear flight control system and achieves early warning. In order to realize the rapid on-line fault detection of the rotor UAV flight system and solve the problems of over-fitting, limited generalization, and long training time in the traditional shallow neural network for sensor fault diagnosis, a comprehensive fault diagnosis method based on deep belief network (DBN) is proposed. Using the DBN to replace the shallow neural network, a large amount of off-line historical sample data obtained from the rotor UAV are trained to obtain the optimal DBN network parameters and complete the on-line intelligent diagnosis to achieve the goal of early warning as possible as quickly. In the end, the two common faults of the UAV sensor, namely the stuck fault and the constant deviation fault, are simulated and compared with the back propagation (BP) neural network model represented by the shallow neural network to verify the effectiveness of the proposed method in the paper.

Three-Dimensional Digital Documentation of Cultural Heritage Site Based on the Convergence of Terrestrial Laser Scanning and Unmanned Aerial Vehicle Photogrammetry

Three-dimensional digital technology is important in the maintenance and monitoring of cultural heritage sites. This study focuses on using a combination of terrestrial laser scanning and unmanned aerial vehicle (UAV) photogrammetry to establish a three-dimensional model and the associated digital documentation of the Magoksa Temple, Republic of Korea. Herein, terrestrial laser scanning and UAV photogrammetry was used to acquire the perpendicular geometry of the buildings and sites, where UAV photogrammetry yielded higher planar data acquisition rate in upper zones, such as the roof of a building, than terrestrial laser scanning. On comparing the two technologies’ accuracy based on their ground control points, laser scanning was observed to provide higher positional accuracy than photogrammetry. The overall discrepancy between the two technologies was found to be sufficient for the generation of convergent data. Thus, the terrestrial laser scanning and UAV photogrammetry data were aligned and merged post conversion into compatible extensions. A three-dimensional (3D) model, with planar and perpendicular geometries, based on the hybrid data-point cloud was developed. This study demonstrates the potential for using the integration of terrestrial laser scanning and UAV photogrammetry in 3D digital documentation and spatial analysis of cultural heritage sites.

Motion-Aware Correlation Filters for Online Visual Tracking

The discriminative correlation filters-based methods struggle deal with the problem of fast motion and heavy occlusion, the problem can severely degrade the performance of trackers, ultimately leading to tracking failures. In this paper, a novel Motion-Aware Correlation Filters (MACF) framework is proposed for online visual object tracking, where a motion-aware strategy based on joint instantaneous motion estimation Kalman filters is integrated into the Discriminative Correlation Filters (DCFs). The proposed motion-aware strategy is used to predict the possible region and scale of the target in the current frame by utilizing the previous estimated 3D motion information. Obviously, this strategy can prevent model drift caused by fast motion. On the base of the predicted region and scale, the MACF detects the position and scale of the target by using the DCFs-based method in the current frame. Furthermore, an adaptive model updating strategy is proposed to address the problem of corrupted models caused by occlusions, where the learning rate is determined by the confidence of the response map. The extensive experiments on popular Object Tracking Benchmark OTB-100, OTB-50 and unmanned aerial vehicles (UAV) video have demonstrated that the proposed MACF tracker performs better than most of the state-of-the-art trackers and achieves a high real-time performance. In addition, the proposed approach can be integrated easily and flexibly into other visual tracking algorithms.

Variables Influencing the Accuracy of 3D Modeling of Existing Roads Using Consumer Cameras in Aerial Photogrammetry

Point cloud (PC) generation from photogrammetry–remotely piloted aircraft systems (RPAS) at high spatial and temporal resolution and accuracy is of increasing importance for many applications. For several years, photogrammetry–RPAS has been used to recover civil engineering works such as digital elevation models (DEMs), triangle irregular networks (TINs), contour levels, orthophotographs, etc. This study analyzes the influence of variables involved in the accuracy of PC generation over asphalt shapes and determines the most influential variable based on the development of an artificial neural network (ANN) with patterns identified in the test flights. The input variables were those involved, and output was the three-dimension root mean square error (3D-RMSE) of the PC in each ground control point (GCP). The result of the study shows that the most influential variable over PC accuracy is the modulation transfer function 50 (MTF50). In addition, the study obtained an average 3D-RMSE of 1 cm. The results can be used by the scientific and civil engineering communities to consider MTF50 variables in obtaining images from RPAS cameras and to predict the accuracy of a PC over asphalt based on the ANN developed. Also, this ANN could be the beginning of a large database containing patterns from several cameras and lenses in the world market.

New Perspectives for UAV-Based Modelling the Roman Gold Mining Infrastructure in NW Spain

This contribution discusses the potential of UAV-assisted (unmanned aerial vehicles) photogrammetry for the study and preservation of mining heritage sites using the example of Roman gold mining infrastructure in northwestern Spain. The study area represents the largest gold area in Roman times and comprises 7 mining elements of interest that characterize the most representative examples of such ancient works. UAV technology provides a non-invasive procedure valuable for the acquisition of digital information in remote, difficult to access areas or under the risk of destruction. The proposed approach is a cost-effective, robust and rapid method for image processing in remote areas were no traditional surveying technologies are available. It is based on a combination of data provided by aerial orthoimage and LiDAR (Light Detection and Ranging) to improve the accuracy of UAV derived data. The results provide high-resolution orthomosaic, DEMs and 3D textured models that aim for the documentation of ancient mining scenarios, providing high-resolution digital information that improves the identification, description and interpretation of mining elements such as the hydraulic infrastructure, the presence of open-cast mines which exemplifies the different exploitation methods, and settlements. However, beyond the scientific and technical information provided by the data, the 3D documentation of ancient mining scenarios is a powerful tool for an effective and wider public diffusion ensuring the visualization, preservation and awareness over the importance and conservation of world mining heritage sites.

A Multi-Disciplinary Approach to Remote Sensing through Low-Cost UAVs

The use of Unmanned Aerial Vehicles (UAVs) based on remote sensing has generated low cost monitoring, since the data can be acquired quickly and easily. This paper reports the experience related to agave crop analysis with a low cost UAV. The data were processed by traditional photogrammetric flow and data extraction techniques were applied to extract new layers and separate the agave plants from weeds and other elements of the environment. Our proposal combines elements of photogrammetry, computer vision, data mining, geomatics and computer science. This fusion leads to very interesting results in agave control. This paper aims to demonstrate the potential of UAV monitoring in agave crops and the importance of information processing with reliable data flow.