Computer Vision-Based Structural Displacement Measurement Robust to Light-Induced Image Degradation for In-Service Bridges

Computer Vision-Based Bridge Inspection and Monitoring: A Review

Bridge inspection and monitoring are usually used to evaluate the status and integrity of bridge structures to ensure their safety and reliability. Computer vision (CV)-based methods have the advantages of being low cost, simple to operate, remote, and non-contact, and have been widely used in bridge inspection and monitoring in recent years. Therefore, this paper reviews three significant aspects of CV-based methods, including surface defect detection, vibration measurement, and vehicle parameter identification. Firstly, the general procedure for CV-based surface defect detection is introduced, and its application for the detection of cracks, concrete spalling, steel corrosion, and multi-defects is reviewed, followed by the robot platforms for surface defect detection. Secondly, the basic principle of CV-based vibration measurement is introduced, followed by the application of displacement measurement, modal identification, and damage identification. Finally, the CV-based vehicle parameter identification methods are introduced and their application for the identification of temporal and spatial parameters, weight parameters, and multi-parameters are summarized. This comprehensive literature review aims to provide guidance for selecting appropriate CV-based methods for bridge inspection and monitoring.

An Image-Based Framework for Measuring the Prestress Level in CFRP Laminates: Experimental Validation

Image-based methods have been applied to support structural monitoring, product and material testing, and quality control. Lately, deep learning for compute vision is the trend, requiring large and labelled datasets for training and validation, which is often difficult to obtain. The use of synthetic datasets is often applying for data augmentation in different fields. An architecture based on computer vision was proposed to measure strain during prestressing in CFRP laminates. The contact-free architecture was fed by synthetic image datasets and benchmarked for machine learning and deep learning algorithms. The use of these data for monitoring real applications will contribute towards spreading the new monitoring approach, increasing the quality control of the material and application procedure, as well as structural safety. In this paper, the best architecture was validated during experimental tests, to evaluate the performance in real applications from pre-trained synthetic data. The results demonstrate that the architecture implemented enables estimating intermediate strain values, i.e., within the range of training dataset values, but it does not allow for estimating strain values outside those range. The architecture allowed for estimating the strain in real images with an error ∼0.5%, higher than that obtained with synthetic images. Finally, it was not possible to estimate the strain in real cases from the training performed with the synthetic dataset.

A Mitigation Method for Optical-Turbulence-Induced Errors and Optimal Target Design in Vision-Based Displacement Measurement

Computer vision-based displacement measurement techniques are increasingly used for structural health monitoring. However, the vision sensors employed are easily affected by optical turbulence when capturing images of the structure, resulting in displacement measurement errors that significantly reduce the accuracy required in engineering applications. Hence, this paper develops a multi-measurement point method to address this problem by mitigating optical-turbulence errors with spatial randomness. Then, the effectiveness of the proposed method in mitigating optical-turbulence errors is verified by static target experiments, in which the RMSE correction rate can reach up to 82%. Meanwhile, the effects of target size and the number of measurement points on the proposed method are evaluated, and the optimal target design criteria are proposed to improve our method's performance in mitigating optical-turbulence errors under different measurement conditions. Additionally, extensive dynamic target experiments reveal that the proposed method achieves an RMSE correction rate of 69% after mitigating the optical-turbulence error. The experimental results demonstrate that the proposed method improves the visual displacement measurement accuracy and retains the detailed information of the displacement measurement results.

A Review of Computer Vision-Based Structural Deformation Monitoring in Field Environments

Computer vision-based structural deformation monitoring techniques were studied in a large number of applications in the field of structural health monitoring (SHM). Numerous laboratory tests and short-term field applications contributed to the formation of the basic framework of computer vision deformation monitoring systems towards developing long-term stable monitoring in field environments. The major contribution of this paper was to analyze the influence mechanism of the measuring accuracy of computer vision deformation monitoring systems from two perspectives, the physical impact, and target tracking algorithm impact, and provide the existing solutions. Physical impact included the hardware impact and the environmental impact, while the target tracking algorithm impact included image preprocessing, measurement efficiency and accuracy. The applicability and limitations of computer vision monitoring algorithms were summarized.

Development and Validation of a Framework for Smart Wireless Strain and Acceleration Sensing

Civil infrastructure worldwide is subject to factors such as aging and deterioration. Structural health monitoring (SHM) can be used to assess the impact of these processes on structural performance. SHM demands have evolved from routine monitoring to real-time and autonomous assessment. One of the frontiers in achieving effective SHM systems has been the use of wireless smart sensors (WSSs), which are attractive compared to wired sensors, due to their flexibility of use, lower costs, and ease of long-term deployment. Most WSSs use accelerometers to collect global dynamic vibration data. However, obtaining local behaviors in a structure using measurands such as strain may also be desirable. While wireless strain sensors have previously been developed by some researchers, there is still a need for a high sensitivity wireless strain sensor that fully meets the general demands for monitoring large-scale civil infrastructure. In this paper, a framework for synchronized wireless high-fidelity acceleration and strain sensing, which is commonly termed multimetric sensing in the literature, is proposed. The framework is implemented on the Xnode, a next-generation wireless smart sensor platform, and integrates with the strain sensor for strain acquisition. An application of the multimetric sensing framework is illustrated for total displacement estimation. Finally, the potential of the proposed framework integrated with vision-based measurement systems for multi-point displacement estimation with camera-motion compensation is demonstrated. The proposed approach is verified experimentally, showing the potential of the developed framework for various SHM applications.

Operational Deflection Shapes Magnification and Visualization Using Optical-Flow-Based Image Processing

Much information can be derived from operational deflection shapes of vibrating structures and the magnification of their motion. However, the acquisition of deflection shapes usually requires a manual definition of an object’s points of interest, while general motion magnification is computationally inefficient. We propose easy extraction of operational deflection shapes straight from vision data by analyzing and processing optical flow information from the video and then, based on these graphs, morphing source data to magnify the shape of deflection. We introduce several processing routines for automatic masking of the optical flow data and frame-wise information fusion. The method is tested based on data acquired both in numerical simulations and real-life experiments in which cantilever beams were subjected to excitation around their natural frequencies.

Vision-Based Approach in Contact Modelling between the Footpad of the Lander and the Analogue Representing Surface of Phobos

Identifying solar system surface properties of celestial bodies requires the conducting of many tests and experiments in conditions similar to those found on various objects. One of the first tasks to be solved by engineers is determining the contact condition between the lander and the surface of a given celestial body during landing in a microgravity environment. This paper presents the results of experimental studies and numerical simulations of the contact phenomenon between the lander foot model and the Phobos analogue. The main goal of the experimental tests was to obtain measured deformation data of the studied analogues using 2D and 3D vision systems, which were employed to analyze the behavior of the lander foot and the surface of the studied analogue itself and to calibrate the numerical models. The analogue representing the Phobos surface was foam concrete. The variable parameters in the study were the analogue thickness and the lander foot velocity at the time of contact. Tests were conducted for three different contact velocities of 1.2 m/s, 3.0 m/s, and 3.5 m/s. Taking into account the mass of the lander foot model, kinetic energies of 30.28 J, 189.22 J, and 257.56 J were obtained. The results showed that at low contact velocities, and thus low kinetic energies, no significant differences in behavior of the material directly under the lander foot were observed, and similar values of forces in the lander foot were obtained. For higher contact velocities, the behavior of analogues with varying thicknesses was different, resulting in different values of analogue deformation and dynamics of increments and decrements of force in the lander foot itself. Although performed on a single material, the experiments revealed different behaviors depending on its thickness at the same impact energy. This is an essential guideline for engineers who need to take this fact into account when designing the lander itself.

Development of a Reference-Free Indirect Bridge Displacement Sensing System

Bridge displacement measurements are important data for assessing the condition of a bridge. Measuring bridge displacement under moving vehicle loads is helpful for rating the load-carrying capacity and evaluating the structural health of a bridge. Displacements are conventionally measured using a linear variable differential transformer (LVDT), which needs stable reference points and thus prohibits the use of this method for measuring displacements for bridges crossing sea channels, large rivers, and highways. This paper proposes a reference-free indirect bridge displacement sensing system using a multichannel sensor board strain and accelerometer with a commercial wireless sensor platform (Xnode). The indirect displacement estimation method is then optimized for measuring the structural displacement. The performance of the developed system was experimentally evaluated on concrete- and steelbox girder bridges. In comparison with the reference LVDT data, the maximum displacement error for the proposed method was 2.17%. The proposed method was successfully applied to the displacement monitoring of a tall bridge (height = 20 m), which was very difficult to monitor using existing systems.

Full-Field Bridge Deflection Monitoring with Off-Axis Digital Image Correlation

Video deflectometer based on using off-axis digital image correlation (DIC) has emerged as a robust non-contact optical tool for deflection measurements of bridges. In practice, a video deflectometer often needs to measure the deflections at multiple positions of the bridge. The existing 2D-DIC-based measurement methods usually use a laser rangefinder to measure the distance from each point to the camera to obtain the scale factor for the point. It is only suitable for the deflection measurements of a few points since manually measuring distances for a large number of points is time consuming and impractical. In this paper, a novel method for full-field bridge deflection measurement based on off-axis DIC is proposed. Because the bridge is usually a slender structure and the region of interest on the bridge is often a narrow band, the new approach can determine the scale factors of all the points of interest with a spatial straight-line fitting scheme. Moreover, the proposed technique employs reliability-guided processing and a fast initial parameter estimation strategy for real-time and accurate image-matching analysis. An indoor cantilever beam experiment verified the accuracy of the proposed approach, and a field test of a high-speed railway bridge demonstrated the robustness and practicability of the technique.

Vision-Based Vibration Monitoring of Structures and Infrastructures: An Overview of Recent Applications

Contactless structural monitoring has in recent years seen a growing number of applications in civil engineering. Indeed, the elimination of physical installations of sensors is very attractive, especially for structures that might not be easily or safely accessible, yet requiring the experimental evaluation of their conditions, for example following extreme events such as strong earthquakes, explosions, and floods. Among contactless technologies, vision-based monitoring is possibly the solution that has attracted most of the interest of civil engineers, given that the advantages of contactless monitoring can be potentially obtained thorough simple and low-cost consumer-grade instrumentations. The objective of this review article is to provide an introductory discussion of the latest applications of vision-based vibration monitoring of structures and infrastructures through an overview of the results achieved in full-scale field tests, as documented in the published technical literature. In this way, engineers new to vision-based monitoring and stakeholders interested in the possibilities of contactless monitoring in civil engineering could have an outline of up-to-date achievements to support a first evaluation of the feasibility and convenience for future monitoring tasks.

An Improved Vision Method for Robust Monitoring of Multi-Point Dynamic Displacements with Smartphones in an Interference Environment

Current research on dynamic displacement measurement based on computer vision mostly requires professional high-speed cameras and an ideal shooting environment to ensure the performance and accuracy of the analysis. However, the high cost of the camera and strict requirements of sharp image contrast and stable environment during the shooting process limit the broad application of the technology. This paper proposes an improved vision method to implement multi-point dynamic displacement measurements with smartphones in an interference environment. A motion-enhanced spatio-temporal context (MSTC) algorithm is developed and applied together with the optical flow (OF) algorithm to realize a simultaneous tracking and dynamic displacement extraction of multiple points on a vibrating structure in the interference environment. Finally, a sine-sweep vibration experiment on a cantilever sphere model is presented to validate the feasibility of the proposed method in a wide-band frequency range. In the test, a smartphone was used to shoot the vibration process of the sine-sweep-excited sphere, and illumination change, fog interference, and camera jitter were artificially simulated to represent the interference environment. The results of the proposed method are compared to conventional displacement sensor data and current vision method results. It is demonstrated that, in an interference environment, (1) the OF method is prone to mismatch the feature points and leads to data deviated or lost; (2) the conventional STC method is sensitive to target selection and can effectively track those targets having a large proportion of pixels in the context with motion tendency similar to the target center; (3) the proposed MSTC method, however, can ease the sensitivity to target selection through in-depth processing of the information in the context and finally enhance the robustness of the target tracking. In addition, the MSTC method takes less than one second to track each target between adjacent frame images, implying a potential for online measurement.

Framework for Structural Health Monitoring of Steel Bridges by Computer Vision

The monitoring of a structural condition of steel bridges is an important issue. Good condition of infrastructure facilities ensures the safety and economic well-being of society. At the same time, due to the continuous development, rising wealth of the society and socio-economic integration of countries, the number of infrastructural objects is growing. Therefore, there is a need to introduce an easy-to-use and relatively low-cost method of bridge diagnostics. We can achieve these benefits by the use of Unmanned Aerial Vehicle-Based Remote Sensing and Digital Image Processing. In our study, we present a state-of-the-art framework for Structural Health Monitoring of steel bridges that involves literature review on steel bridges health monitoring, drone route planning, image acquisition, identification of visual markers that may indicate a poor condition of the structure and determining the scope of applicability. The presented framework of image processing procedure is suitable for diagnostics of steel truss riveted bridges. In our considerations, we used photographic documentation of the Fitzpatrick Bridge located in Tallassee, Alabama, USA.

A Sequential Framework for Improving Identifiability of FE Model Updating using Static and Dynamic Data

By virtue of the advances in sensing techniques, finite element (FE) model updating (FEMU) using static and dynamic data has been recently employed to improve identification on updating parameters. Using heterogeneous data can provide useful information to improve parameter identifiability in FEMU. It is worth noting that the useful information from the heterogeneous data may be diluted in the conventional FEM framework. The conventional FEMU framework in previous studies have used heterogeneous data at once to compute residuals in the objective function, and they are condensed to be a scalar. In this implementation, it should be careful to formulate the objective function with proper weighting factors to consider the scale of measurement and relative significances. Otherwise, the information from heterogeneous data cannot be efficiently utilized. For FEMU of the bridge, parameter compensation may exist due to mutual dependence among updating parameters. This aggravates the parameter identifiability to make the results of the FEMU worse. To address the limitation of the conventional FEMU method, this study proposes a sequential framework for the FEMU of existing bridges. The proposed FEMU method uses two steps to utilize static and dynamic data in a sequential manner. By using them separately, the influence of the parameter compensation can be suppressed. The proposed FEMU method is verified through numerical and experimental study. Through these verifications, the limitation of the conventional FEMU method is investigated in terms of parameter identifiability and predictive performance. The proposed FEMU method shows much smaller variabilities in the updating parameters than the conventional one by providing the better predictions than those of the conventional one in calibration and validation data. Based on numerical and experimental study, the proposed FEMU method can improve the parameter identifiability using the heterogeneous data and it seems to be promising and efficient framework for FEMU of the existing bridge.

Measurement of Three-Dimensional Structural Displacement Using a Hybrid Inertial Vision-Based System

Accurate three-dimensional displacement measurements of bridges and other structures have received significant attention in recent years. The main challenges of such measurements include the cost and the need for a scalable array of instrumentation. This paper presents a novel Hybrid Inertial Vision-Based Displacement Measurement (HIVBDM) system that can measure three-dimensional structural displacements by using a monocular charge-coupled device (CCD) camera, a stationary calibration target, and an attached tilt sensor. The HIVBDM system does not require the camera to be stationary during the measurements, while the camera movements, i.e., rotations and translations, during the measurement process are compensated by using a stationary calibration target in the field of view (FOV) of the camera. An attached tilt sensor is further used to refine the camera movement compensation, and better infers the global three-dimensional structural displacements. This HIVBDM system is evaluated on both short-term and long-term synthetic static structural displacements, which are conducted in an indoor simulated experimental environment. In the experiments, at a 9.75 m operating distance between the monitoring camera and the structure that is being monitored, the proposed HIVBDM system achieves an average of 1.440 mm Root Mean Square Error (RMSE) on the in-plane structural translations and an average of 2.904 mm RMSE on the out-of-plane structural translations.

Non-Target Structural Displacement Measurement Using Reference Frame-Based Deepflow

Displacement is crucial for structural health monitoring, although it is very challenging to measure under field conditions. Most existing displacement measurement methods are costly, labor-intensive, and insufficiently accurate for measuring small dynamic displacements. Computer vision (CV)-based methods incorporate optical devices with advanced image processing algorithms to accurately, cost-effectively, and remotely measure structural displacement with easy installation. However, non-target-based CV methods are still limited by insufficient feature points, incorrect feature point detection, occlusion, and drift induced by tracking error accumulation. This paper presents a reference frame-based Deepflow algorithm integrated with masking and signal filtering for non-target-based displacement measurements. The proposed method allows the user to select points of interest for images with a low gradient for displacement tracking and directly calculate displacement without drift accumulated by measurement error. The proposed method is experimentally validated on a cantilevered beam under ambient and occluded test conditions. The accuracy of the proposed method is compared with that of a reference laser displacement sensor for validation. The significant advantage of the proposed method is its flexibility in extracting structural displacement in any region on structures that do not have distinct natural features.

Long-Term Deflection Prediction from Computer Vision-Measured Data History for High-Speed Railway Bridges

Management of the vertical long-term deflection of a high-speed railway bridge is a crucial factor to guarantee traffic safety and passenger comfort. Therefore, there have been efforts to predict the vertical deflection of a railway bridge based on physics-based models representing various influential factors to vertical deflection such as concrete creep and shrinkage. However, it is not an easy task because the vertical deflection of a railway bridge generally involves several sources of uncertainty. This paper proposes a probabilistic method that employs a Gaussian process to construct a model to predict the vertical deflection of a railway bridge based on actual vision-based measurement and temperature. To deal with the sources of uncertainty which may cause prediction errors, a Gaussian process is modeled with multiple kernels and hyperparameters. Once the hyperparameters are identified through the Gaussian process regression using training data, the proposed method provides a 95% prediction interval as well as a predictive mean about the vertical deflection of the bridge. The proposed method is applied to an arch bridge under operation for high-speed trains in South Korea. The analysis results obtained from the proposed method show good agreement with the actual measurement data on the vertical deflection of the example bridge, and the prediction results can be utilized for decision-making on railway bridge maintenance.

A Novel Laser and Video-Based Displacement Transducer to Monitor Bridge Deflections

The measurement of static vertical deflections on bridges continues to be a first-level technological challenge. These data are of great interest, especially for the case of long-term bridge monitoring; in fact, they are perhaps more valuable than any other measurable parameter. This is because material degradation processes and changes of the mechanical properties of the structure due to aging (for example creep and shrinkage in concrete bridges) have a direct impact on the exhibited static vertical deflections. This paper introduces and evaluates an approach to monitor displacements and rotations of structures using a novel laser and video-based displacement transducer (LVBDT). The proposed system combines the use of laser beams, LED lights, and a digital video camera, and was especially designed to capture static and slow-varying displacements. Contrary to other video-based approaches, the camera is located on the bridge, hence allowing to capture displacements at one location. Subsequently, the sensing approach and the procedure to estimate displacements and the rotations are described. Additionally, laboratory and in-service field testing carried out to validate the system are presented and discussed. The results demonstrate that the proposed sensing approach is robust, accurate, and reliable, and also inexpensive, which are essential for field implementation.