An Image-Based Approach for the Co-Registration of Multi-Temporal UAV Image Datasets

Post Disaster Damage Assessment Using Ultra-High-Resolution Aerial Imagery with Semi-Supervised Transformers

Preliminary damage assessments (PDA) conducted in the aftermath of a disaster are a key first step in ensuring a resilient recovery. Conventional door-to-door inspection practices are time-consuming and may delay governmental resource allocation. A number of research efforts have proposed frameworks to automate PDA, typically relying on data sources from satellites, unmanned aerial vehicles, or ground vehicles, together with data processing using deep convolutional neural networks. However, before such frameworks can be adopted in practice, the accuracy and fidelity of predictions of damage level at the scale of an entire building must be comparable to human assessments. Towards this goal, we propose a PDA framework leveraging novel ultra-high-resolution aerial (UHRA) images combined with state-of-the-art transformer models to make multi-class damage predictions of entire buildings. We demonstrate that semi-supervised transformer models trained with vast amounts of unlabeled data are able to surpass the accuracy and generalization capabilities of state-of-the-art PDA frameworks. In our series of experiments, we aim to assess the impact of incorporating unlabeled data, as well as the use of different data sources and model architectures. By integrating UHRA images and semi-supervised transformer models, our results suggest that the framework can overcome the significant limitations of satellite imagery and traditional CNN models, leading to more accurate and efficient damage assessments.

WELDMAP: A Photogrammetric Suite Applied to the Inspection of Welds

This paper presents a new tool for external quality control in welds using close-range photogrammetry. The main contribution of the developed approach is the automatic assessment of welds based on 3D photogrammetric models, enabling objective and accurate analyses through an in-house tool that was developed, WELDMAP. As a result, inspectors can perform external quality control of welds in a simple and efficient way without requiring visual inspections or external tools, and thus avoiding the subjectivity and imprecisions of the classical protocol. The tool was validated with a large dataset in laboratory tests as well as in real scenarios.

Boosting the Timeliness of UAV Large Scale Mapping. Direct Georeferencing Approaches: Operational Strategies and Best Practices

The use of unmanned aerial vehicles (UAVs) is nowadays a standard approach in several application fields. Researches connected with these systems cover several topics and the evolution of these platforms and their applications are rapidly growing. Despite the high level of automatization reached nowadays, there is still a phase of the overall UAVs’ photogrammetric pipeline that requires a high effort in terms of time and resources (i.e., the georeferencing phase). However, thanks to the availability of survey-grade GNSS (Global Navigation Satellite System) receivers embedded in the aerial platforms, it is possible to also enhance this phase of the processing by adopting direct georeferencing approaches (i.e., without using any ground control point and exploiting real time kinematic (RTK) positioning). This work investigates the possibilities offered by a multirotor commercial system equipped with a RTK-enabled GNSS receiver, focusing on the accuracy of the georeferencing phase. Several tests were performed in an ad-hoc case study exploiting different georeferencing solutions and assessing the 3D positional accuracies, thanks to a network of control points. The best approaches to be adopted in the field according to accuracy requirements of the final map products were identified and operational guidelines proposed accordingly.

Geo-Registering Consecutive Construction Site Recordings Using a Pre-Registered Reference Module

The monitoring of construction sites and the achieved progress is a vital aspect of construction works in the Architecture, Engineering and Construction (AEC) industry. Only if three-dimensional reconstructions require a limited time, the creation of consecutive datasets, portraying the site in subsequent phases, becomes feasible. Moreover, a shared coordinate system between all datasets is essential for monitoring purposes. In this work, a new photogrammetric framework is presented to shift from the current error-prone and tedious manual geo-referencing process to a semi-automated one. The fundament of the method is an accurately processed reference module that repeatedly serves as the starting point for processing subsequent image datasets. By means of overlap between pictures in both datasets, the coordinate system, incorporated in the reference module, is inherited by the subsequent datasets, hence bypassing the indication process. The proposed procedure is able to outperform current methods, while requiring less time considered over all consecutive datasets. In our experiments, we compared an unaltered part of two subsequent datasets, each of them processed via the traditional and our proposed method. The obtained mean disparity was 9 mm, while for the manual approach it was 16 mm. Especially for comparative analyses, the proposed approach yields excellent results as every dataset is registered exactly the same, whereas results diverge more when following manual methods. In conclusion, our approach is favourable over the current one, especially for a multitude of consecutive site reconstructions, as no ground control points (GCPs) must be indicated in each separate subsequent dataset, while yielding similar to even better results.

Spatial Estimation of the Latent Heat Flux in a Tropical Dry Forest by Using Unmanned Aerial Vehicles

In this paper, we address the retrieval of spatially distributed latent heat flux ( λ E) over a tropical dry forest using multi-spectral and thermal unmanned aerial vehicle (UAV) imagery. The study was carried out in the Santa Rosa National Park Environmental Monitoring Super-Site, Costa Rica, in June 2016. The triangle method was used to derive λ E from the UAV imagery and the results were compared to λ E measurements of an eddy covariance system within the coincident eddy flux tower footprint. The tower footprint was derived using a two-dimensional parameterization model for flux footprint prediction. The comparisons with the flux tower measurements showed a mean relative difference of 10.98% with a slight overestimation of the UAV-based flux retrievals by nearly 7.7 Wm − 2 . The results are in good agreement with satellite-based retrievals, as provided by the literature, for which the triangle method was initially developed and mostly used so far. This study proved to be a promising approach for transferring the triangle method to UAV imagery in ecosystems such as tropical dry forests. With the presented approach, new details in spatially distributed latent heat flux estimates at ultra-high resolution are now possible, thereby potentially closing the gap in spatial resolution between satellites and flux towers. Even more, it allows tracing the latent heat flux from single trees at leaf level. Besides, this approach also opens new perspectives for the monitoring of latent heat fluxes in tropical dry forests.

Improved Piecewise Linear Transformation for Precise Warping of Very-High-Resolution Remote Sensing Images

A large number of evenly distributed conjugate points (CPs) in entirely overlapping regions of the images are required to achieve successful co-registration between very-high-resolution (VHR) remote sensing images. The CPs are then used to construct a non-linear transformation model that locally warps a sensed image to a reference image’s coordinates. Piecewise linear (PL) transformation is largely exploited for warping VHR images because of its superior performance as compared to the other methods. The PL transformation constructs triangular regions on a sensed image from the CPs by applying the Delaunay algorithm, after which the corresponding triangular regions in a reference image are constructed using the same CPs on the image. Each corresponding region in the sensed image is then locally warped to the regions of the reference image through an affine transformation estimated from the CPs on the triangle vertices. The warping performance of the PL transformation shows reliable results, particularly in regions inside the triangles, i.e., within the convex hulls. However, the regions outside the triangles, which are warped when the extrapolated boundary planes are extended using CPs located close to the regions, incur severe geometric distortion. In this study, we propose an effective approach that focuses on the improvement of the warping performance of the PL transformation over the external area of the triangles. Accordingly, the proposed improved piecewise linear (IPL) transformation uses additional pseudo-CPs intentionally extracted from positions on the boundary of the sensed image. The corresponding pseudo-CPs on the reference image are determined by estimating the affine transformation from CPs located close to the pseudo-CPs. The latter are simultaneously used with the former to construct the triangular regions, which are enlarged accordingly. Experiments on both simulated and real datasets, constructed from Worldview-3 and Kompsat-3A satellite images, were conducted to validate the effectiveness of the proposed IPL transformation. That transformation was shown to outperform the existing linear/non-linear transformation models such as an affine, third and fourth polynomials, local weighted mean, and PL. Moreover, we demonstrated that the IPL transformation improved the warping performance over the PL transformation outside the triangular regions by increasing the correlation coefficient values from 0.259 to 0.304, 0.603 to 0.657, and 0.180 to 0.338 in the first, second, and third real datasets, respectively.

New Target for Accurate Terrestrial Laser Scanning and Unmanned Aerial Vehicle Point Cloud Registration

The main goal of our research was to design and implement an innovative target that would be suitable for accurately registering point clouds produced from unmanned aerial vehicle (UAV) images and terrestrial laser scans. Our new target is composed of three perpendicular planes that combine the properties of plane and volume targets. The new target enables the precise determination of reference target points in aerial and terrestrial point clouds. Different types of commonly used plane and volume targets as well as the new target were placed in an established test area in order to evaluate their performance. The targets were scanned from multiple scanner stations and surveyed with an unmanned aerial vehicle DJI Phantom 4 PRO at three different altitudes (20, 40, and 75 m). The reference data were measured with a Leica Nova MS50 MultiStation. Several registrations were performed, each time with a different target. The quality of these registrations was assessed on the check points. The results showed that the new target yielded the best results in all cases, which confirmed our initial expectations. The proposed new target is innovative and not difficult to create and use.

Speed Estimation of Multiple Moving Objects from a Moving UAV Platform

Speed detection of a moving object using an optical camera has always been an important subject to study in computer vision. This is one of the key components to address in many application areas, such as transportation systems, military and naval applications, and robotics. In this study, we implemented a speed detection system for multiple moving objects on the ground from a moving platform in the air. A detect-and-track approach is used for primary tracking of the objects. Faster R-CNN (region-based convolutional neural network) is applied to detect the objects, and a discriminative correlation filter with CSRT (channel and spatial reliability tracking) is used for tracking. Feature-based image alignment (FBIA) is done for each frame to get the proper object location. In addition, SSIM (structural similarity index measurement) is performed to check how similar the current frame is with respect to the object detection frame. This measurement is necessary because the platform is moving, and new objects may be captured in a new frame. We achieved a speed accuracy of 96.80% with our framework with respect to the real speed of the objects.

Rapid Mosaicking of Unmanned Aerial Vehicle (UAV) Images for Crop Growth Monitoring Using the SIFT Algorithm

To improve the efficiency and effectiveness of mosaicking unmanned aerial vehicle (UAV) images, we propose in this paper a rapid mosaicking method based on scale-invariant feature transform (SIFT) for mosaicking UAV images used for crop growth monitoring. The proposed method dynamically sets the appropriate contrast threshold in the difference of Gaussian (DOG) scale-space according to the contrast characteristics of UAV images used for crop growth monitoring. Therefore, this method adjusts and optimizes the number of matched feature point pairs in UAV images and increases the mosaicking efficiency. Meanwhile, based on the relative location relationship of UAV images used for crop growth monitoring, the random sample consensus (RANSAC) algorithm is integrated to eliminate the influence of mismatched point pairs in UAV images on mosaicking and to keep the accuracy and quality of mosaicking. Mosaicking experiments were conducted by setting three types of UAV images in crop growth monitoring: visible, near-infrared, and thermal infrared. The results indicate that compared to the standard SIFT algorithm and frequently used commercial mosaicking software, the method proposed here significantly improves the applicability, efficiency, and accuracy of mosaicking UAV images in crop growth monitoring. In comparison with image mosaicking based on the standard SIFT algorithm, the time efficiency of the proposed method is higher by 30%, and its structural similarity index of mosaicking accuracy is about 0.9. Meanwhile, the approach successfully mosaics low-resolution UAV images used for crop growth monitoring and improves the applicability of the SIFT algorithm, providing a technical reference for UAV application used for crop growth and phenotypic monitoring.

Mapping Cultural Heritage in Coastal Areas with UAS: The Case Study of Lesvos Island

Dynamic processes in coastal zones and human activities in the coastal environment produce pressure on cultural heritage, especially in touristic places. Unmanned aerial systems (UAS) are used as an additional tool for monitoring cultural heritage sites in sensitive coastal areas. UASs provide low-cost accurate spatial data and high-resolution imagery products in various spatial and temporal scales. The use of UAS for mapping cultural heritage sites in the coastal zone is of increasing interest among scientists and archaeologists in terms of monitoring, documentation, mapping, and restoration. This study outlines the integration of UAS data acquisition and structure from motion (SfM) pipeline for the visualization of selected cultural heritage areas (ancient harbors) in the coastal zone. The UAS-SfM methodology produces very detailed orthophoto maps for mapping and detecting cultural heritage sites. Additionally, a metadata cataloging system has been developed in order to facilitate online searching operations for all products of the data acquisition, SfM pipeline, and cartographic processes. For this reason, a specific metadata profile was implemented, based on the European INSPIRE framework. As a result, datasets reusability and catalogs interoperability are promoted.

Automated Geo/Co-Registration of Multi-Temporal Very-High-Resolution Imagery

For time-series analysis using very-high-resolution (VHR) multi-temporal satellite images, both accurate georegistration to the map coordinates and subpixel-level co-registration among the images should be conducted. However, applying well-known matching methods, such as scale-invariant feature transform and speeded up robust features for VHR multi-temporal images, has limitations. First, they cannot be used for matching an optical image to heterogeneous non-optical data for georegistration. Second, they produce a local misalignment induced by differences in acquisition conditions, such as acquisition platform stability, the sensor’s off-nadir angle, and relief displacement of the considered scene. Therefore, this study addresses the problem by proposing an automated geo/co-registration framework for full-scene multi-temporal images acquired from a VHR optical satellite sensor. The proposed method comprises two primary steps: (1) a global georegistration process, followed by (2) a fine co-registration process. During the first step, two-dimensional multi-temporal satellite images are matched to three-dimensional topographic maps to assign the map coordinates. During the second step, a local analysis of registration noise pixels extracted between the multi-temporal images that have been mapped to the map coordinates is conducted to extract a large number of well-distributed corresponding points (CPs). The CPs are finally used to construct a non-rigid transformation function that enables minimization of the local misalignment existing among the images. Experiments conducted on five Kompsat-3 full scenes confirmed the effectiveness of the proposed framework, showing that the georegistration performance resulted in an approximately pixel-level accuracy for most of the scenes, and the co-registration performance further improved the results among all combinations of the georegistered Kompsat-3 image pairs by increasing the calculated cross-correlation values.

Unmanned Aerial Vehicle Systems for Remote Estimation of Flooded Areas Based on Complex Image Processing

Floods are natural disasters which cause the most economic damage at the global level. Therefore, flood monitoring and damage estimation are very important for the population, authorities and insurance companies. The paper proposes an original solution, based on a hybrid network and complex image processing, to this problem. As first novelty, a multilevel system, with two components, terrestrial and aerial, was proposed and designed by the authors as support for image acquisition from a delimited region. The terrestrial component contains a Ground Control Station, as a coordinator at distance, which communicates via the internet with more Ground Data Terminals, as a fixed nodes network for data acquisition and communication. The aerial component contains mobile nodes—fixed wing type UAVs. In order to evaluate flood damage, two tasks must be accomplished by the network: area coverage and image processing. The second novelty of the paper consists of texture analysis in a deep neural network, taking into account new criteria for feature selection and patch classification. Color and spatial information extracted from chromatic co-occurrence matrix and mass fractal dimension were used as well. Finally, the experimental results in a real mission demonstrate the validity of the proposed methodologies and the performances of the algorithms.