Measurement of Three-Dimensional Structural Displacement Using a Hybrid Inertial Vision-Based System

Drone-based displacement measurement of infrastructures utilizing phase information

Drone-based inspections provide an efficient and flexible approach to assessing aging infrastructures while prioritizing safety. Here, we present a pioneering framework that employs drone cameras for high-precision displacement measurement and achieves sub-millimeter accuracy, meeting the requirements for on-site inspections. Inspired by the principles of human auditory equilibrium, we have developed an effective scheme using a group of strategical reference markers on the bridge girders to measure structural displacements in the bridge. Our approach integrates the phase-based sampling moiré technique with four degrees-of-freedom geometric modeling to accurately delineate the desired bridge displacements from camera motion-induced displacements. The proposed scheme demonstrates favorable precision with accuracy reaching up to 1/100th of a pixel. Real-world validations further confirmed the reliability and efficiency of this technique, making it a practical tool for bridge displacement measurement. Beyond its current applications, this methodology holds promise as a foundational element in shaping the landscape of future autonomous infrastructure inspection systems.

A Review of Vision-Laser-Based Civil Infrastructure Inspection and Monitoring

Structural health and construction security are important problems in civil engineering. Regular infrastructure inspection and monitoring methods are mostly performed manually. Early automatic structural health monitoring techniques were mostly based on contact sensors, which usually are difficult to maintain in complex infrastructure environments. Therefore, non-contact infrastructure inspection and monitoring techniques received increasing interest in recent years, and they are widely used in all aspects of infrastructure life, owing to their convenience and non-destructive properties. This paper provides an overview of vision-based inspection and vision-laser-based monitoring techniques and applications. The inspection part includes image-processing algorithms, object detection, and semantic segmentation. In particular, infrastructure monitoring involves not only visual technologies but also different fusion methods of vision and lasers. Furthermore, the most important challenges for future automatic non-contact inspections and monitoring are discussed and the paper correspondingly concludes with state-of-the-art algorithms and applications to resolve these challenges.

Long-Periodic Analysis of Boresight Misalignment of Ziyuan3-01 Three-Line Camera

The Ziyuan3-01 (ZY3-01) satellite is China’s first civilian stereo surveying and mapping satellite to meet the 1:50,000 scale mapping requirements, and has been operated in orbit for 10 years. The boresight misalignment of the three-line camera (TLC) is an essential factor affecting the geolocation accuracy, which is a principal concern for stereo mapping satellites. However, the relative relationships of TLC are often regarded as fixed for the same ground scene in most traditional geometric calibrations, without considering the on-orbit long-periodic changes. In this paper, we propose a long-periodic method to analyze and estimate the boresight misalignments between three cameras, with the attitude estimation of a nadir (NAD) camera as the benchmark. Offsets and drifts of the three cameras were calculated and calibrated with different compensation models using scale invariant feature transform (SIFT) points as the ground control. Ten simultaneous NAD–Forward (FWD)–Backward (BWD) imagery of the ZY3-01 satellite acquired from 2012 to 2020 were selected to verify the long-periodic changes in TLC boresight misalignments. The results indicate that the boresight alignment angles of ZY3-01 TLC are dynamic during the long-periodic flight, but the structure of TLC is stable for the misalignments of both FWD and BWD within only 7 arc seconds, which can provide a positive reference for subsequent satellite design and long-periodic on-orbit geometric calibration.

Probabilistic Modeling of Motion Blur for Time-of-Flight Sensors

Synthetically creating motion blur in two-dimensional (2D) images is a well-understood process and has been used in image processing for developing deblurring systems. There are no well-established techniques for synthetically generating arbitrary motion blur within three-dimensional (3D) images, such as depth maps and point clouds since their behavior is not as well understood. As a prerequisite, we have previously developed a method for generating synthetic motion blur in a plane that is parallel to the sensor detector plane. In this work, as a major extension, we generalize our previously developed framework for synthetically generating linear and radial motion blur along planes that are at arbitrary angles with respect to the sensor detector plane. Our framework accurately captures the behavior of the real motion blur that is encountered using a Time-of-Flight (ToF) sensor. This work uses a probabilistic model that predicts the location of invalid pixels that are typically present within depth maps that contain real motion blur. More specifically, the probabilistic model considers different angles of motion paths and the velocity of an object with respect to the image plane of a ToF sensor. Extensive experimental results are shown that demonstrate how our framework can be applied to synthetically create radial, linear, and combined radial-linear motion blur. We quantify the accuracy of the synthetic generation method by comparing the resulting synthetic depth map to the experimentally captured depth map with motion. Our results indicate that our framework achieves an average Boundary F1 (BF) score of 0.7192 for invalid pixels for synthetic radial motion blur, an average BF score of 0.8778 for synthetic linear motion blur, and an average BF score of 0.62 for synthetic combined radial-linear motion blur.

CFNet: LiDAR-Camera Registration Using Calibration Flow Network

As an essential procedure of data fusion, LiDAR-camera calibration is critical for autonomous vehicles and robot navigation. Most calibration methods require laborious manual work, complicated environmental settings, and specific calibration targets. The targetless methods are based on some complex optimization workflow, which is time-consuming and requires prior information. Convolutional neural networks (CNNs) can regress the six degrees of freedom (6-DOF) extrinsic parameters from raw LiDAR and image data. However, these CNN-based methods just learn the representations of the projected LiDAR and image and ignore the correspondences at different locations. The performances of these CNN-based methods are unsatisfactory and worse than those of non-CNN methods. In this paper, we propose a novel CNN-based LiDAR-camera extrinsic calibration algorithm named CFNet. We first decided that a correlation layer should be used to provide matching capabilities explicitly. Then, we innovatively defined calibration flow to illustrate the deviation of the initial projection from the ground truth. Instead of directly predicting the extrinsic parameters, we utilize CFNet to predict the calibration flow. The efficient Perspective-n-Point (EPnP) algorithm within the RANdom SAmple Consensus (RANSAC) scheme is applied to estimate the extrinsic parameters with 2D–3D correspondences constructed by the calibration flow. Due to its consideration of the geometric information, our proposed method performed better than the state-of-the-art CNN-based methods on the KITTI datasets. Furthermore, we also tested the flexibility of our approach on the KITTI360 datasets.