Motion microscopy for visualizing and quantifying small motions

Advanced Video-Based Processing for Low-Cost Damage Assessment of Buildings under Seismic Loading in Shaking Table Tests

This paper explores the potential of a low-cost, advanced video-based technique for the assessment of structural damage to buildings caused by seismic loading. A low-cost, high-speed video camera was utilized for the motion magnification processing of footage of a two-story reinforced-concrete frame building subjected to shaking table tests. The damage after seismic loading was estimated by analyzing the dynamic behavior (i.e., modal parameters) and the structural deformations of the building in magnified videos. The results using the motion magnification procedure were compared for validation of the method of the damage assessment obtained through analyses of conventional accelerometric sensors and high-precision optical markers tracked using a passive 3D motion capture system. In addition, 3D laser scanning to obtain an accurate survey of the building geometry before and after the seismic tests was carried out. In particular, accelerometric recordings were also processed and analyzed using several stationary and nonstationary signal processing techniques with the aim of analyzing the linear behavior of the undamaged structure and the nonlinear structural behavior during damaging shaking table tests. The proposed procedure based on the analysis of magnified videos provided an accurate estimate of the main modal frequency and the damage location through the analysis of the modal shapes, which were confirmed using advanced analyses of the accelerometric data. Consequently, the main novelty of the study was the highlighting of a simple procedure with high potential for the extraction and analysis of modal parameters, with a special focus on the analysis of the modal shape's curvature, which provides accurate information on the location of the damage in a structure, while using a noncontact and low-cost method.

Motion Magnification Applications for the Protection of Italian Cultural Heritage Assets

In recent years, the ENEA has introduced a novel methodology based on motion magnification (MM) into the Italian cultural heritage protection and monitoring field. It consists of a digital video signal processing technique able to amplify enormously the tiny movements recorded in conventional videos, while preserving the general topology of the acquired frames. Though the idea of such a methodology is not new, it has recently been provided with an efficient algorithm that makes possible a viable and low-cost magnification. Applications are extremely varied in almost every field of science and technology; however, we are interested in its application to the safeguarding of architectural heritage, a sector of the utmost importance for Italy. As ancient buildings can be extremely sensitive to even minimally invasive instrumentation, most common monitoring sensors can be replaced by contactless tools and methods, such as video-based techniques like MM. It offers many advantages: easy to use, contactless devices, virtual sensors, reusability of the videos, practicality, intuitive graphical results, quantitative analyses capability and low costs. These characteristics are well suited to the monitoring of large ancient monuments; on the other hand, historical sites have peculiarities of their own, requiring careful approaches, proper tools and trained personnel. Moreover, outdoor applications of MM present quite notable difficulties from a practical point of view, e.g., the dimensions of the studied objects, uncontrolled environmental conditions, spurious vibrations, lighting change/instability, etc. Here we give a general idea of the potential of MM and related issues, using some relevant in-the-field case studies in Italian heritage protection.

Vision-Based Structural Modal Identification Using Hybrid Motion Magnification

As a promising alternative to conventional contact sensors, vision-based technologies for a structural dynamic response measurement and health monitoring have attracted much attention from the research community. Among these technologies, Eulerian video magnification has a unique capability of analyzing modal responses and visualizing modal shapes. To reduce the noise interference and improve the quality and stability of the modal shape visualization, this study proposes a hybrid motion magnification framework that combines linear and phase-based motion processing. Based on the assumption that temporal variations can represent spatial motions, the linear motion processing extracts and manipulates the temporal intensity variations related to modal responses through matrix decomposition and underdetermined blind source separation (BSS) techniques. Meanwhile, the theory of Fourier transform profilometry (FTP) is utilized to reduce spatial high-frequency noise. As all spatial motions in a video are linearly controllable, the subsequent phase-based motion processing highlights the motions and visualizes the modal shapes with a higher quality. The proposed method is validated by two laboratory experiments and a field test on a large-scale truss bridge. The quantitative evaluation results with high-speed cameras demonstrate that the hybrid method performs better than the single-step phase-based motion magnification method in visualizing sound-induced subtle motions. In the field test, the vibration characteristics of the truss bridge when a train is driving across the bridge are studied with a commercial camera over 400 m away from the bridge. Moreover, four full-field modal shapes of the bridge are successfully observed.

All-fiber heterodyne velocity and displacement interferometer based on DPLL Doppler tracking with sub-nanometer per second and picometer sensitivity

Velocity and displacement measurements play an important role not only in the process of industrial production and metrology on the ground but also in satellite gravity measurement in space. A high-precision all-fiber heterodyne velocity and displacement interferometer based on digital phase-locked loop (DPLL) Doppler tracking is proposed in this paper. The target velocity is measured by tracking the heterodyne frequency changes of the beat-note signal, and the displacement is obtained by the integrated phase of the Doppler frequency change. A dual-signal differential optical-path scheme combined with DPLL signal tracking technology enables high-precision and high-linearity measurement of velocity and displacement simultaneously. For integration and compactness, the interferometer uses all-fiber optics that are packaged in a small box with dimensions of 150×150×70m m ³, except for an externally fiber-connected collimator as the sensor head. The experimental results show a velocity sensitivity below 30p m/s/H z ^1/2 in the 0.03-2 Hz band and a displacement sensitivity below 10p m/H z ^1/2 above 0.4 Hz.

Label-free detection of leukemic myeloblasts in hyaluronic acid

Chronic myeloid leukemia is generally required bone marrow biopsy for diagnosis. Although examining peripheral blood is less invasive, it has not been fully validated as a routine diagnostic test due to suboptimal sensitivity. To overcome this limitation, a number of methodologies based on microfluidics have been developed for sorting circulating tumor cells from peripheral blood of patients with leukemia.

In order to develop a more convenient method, we designed an analysis protocol using motion microscopy that amplifies cellular micro motions in a captured video by re-rendering pixels to generate extreme magnified visuals. Intriguingly, no fluctuations around leukemic myeloblasts were observed with a motion microscope at any wavelength of 0–10 Hz. However, use of 0.05% hyaluronic acid, one type of non-newtonian fluid, demonstrated fluctuations around leukemic myeloblasts under conditions of 25 μm/s and 0.5–1.5 Hz with a motion microscope.

Thus, the non-invasive detection of leukemic myeloblasts can offer a valuable supplementary diagnostic tool for assessment of drug efficacy for monitoring patients with chronic myeloid leukemia.

Hybrid-driven structural modal shape visualization using subtle variations in high-speed video

The phase-based motion magnification technique can exaggerate specific structural vibrations and obtain potential applications in visualizing and understanding modal shapes. However, the quality of motion magnification is affected by noise and clipping artifacts, especially in large amplifications. We propose a hybrid-driven motion magnification framework that combines Eulerian and Lagrangian motion processing. Since the structural global spatial vibration corresponding to different modal shapes usually accumulates energy differences in the timeline, from a Eulerian perspective, temporal intensity variations are denoised and separated according to the energy distribution to control spatial motions. Meanwhile, from a Lagrangian perspective, the motion magnification is realized by compensating spatial motion according to the magnified inter-frame motion vector field. By utilizing both Eulerian and Lagrangian motion processing, the proposed framework supports a larger amplification factor and achieves better performance in perceiving subtle vibrations in controlled modal tests.

Homodyne laser vibrometer modified by an LCVR for measurement at the nanometer level

The existence of periodic nonlinear error restricts the performance of the homodyne laser vibrometer in sub-fringe amplitude vibration measurement. A homodyne laser vibrometer with nanoscale-amplitude detectability by using a liquid crystal variable retarder (LCVR) is proposed. The LCVR introduces an extra variation of optical path difference larger than the laser wavelength to acquire a full ellipse so that the nonlinearity correction parameters could be pre-extracted. The experiments showed that the nonlinear error could be well suppressed with the correction process based on the pre-extracted parameters, and the detectable minimum amplitude is less than 1 nm. In addition, measurement of vibration with the reflectivity of measured targets down to 0.048% was achieved with an automatic-gain-control module.

Operational Deflection Shapes Magnification and Visualization Using Optical-Flow-Based Image Processing

Much information can be derived from operational deflection shapes of vibrating structures and the magnification of their motion. However, the acquisition of deflection shapes usually requires a manual definition of an object’s points of interest, while general motion magnification is computationally inefficient. We propose easy extraction of operational deflection shapes straight from vision data by analyzing and processing optical flow information from the video and then, based on these graphs, morphing source data to magnify the shape of deflection. We introduce several processing routines for automatic masking of the optical flow data and frame-wise information fusion. The method is tested based on data acquired both in numerical simulations and real-life experiments in which cantilever beams were subjected to excitation around their natural frequencies.

Motion Magnification of Vibration Image in Estimation of Technical Object Condition-Review

One of the most important features of the proper operation of technical objects is monitoring the vibrations of their mechanical components. The currently significant proportion of the research methods in this regard includes a group of research methods based on the conversion of vibrations using sensors providing data from individual locations. In parallel with the continuous improvement of these tools, new methods for acquiring information on the condition of the object have emerged due to the rapid development of visual systems. Their actual effectiveness determined the switch from research laboratories to actual industrial installations. In many cases, the application of the visualization methods can supplement the conventional methods applied and, under particular conditions, can effectively replace them. The decisive factor is their non-contact nature and the possibility for simultaneous observation of multiple points of the selected area. Visual motion magnification (MM) is an image processing method that involves the conscious and deliberate deformation of input images to the form that enables the visual observation of vibration processes which are not visible in their natural form. The first part of the article refers to the basic terms in the field of expressing motion in an image (based on the Lagrangian and Eulerian approaches), the formulation of the term of optical flow (OF), and the interpretation of an image in time and space. The following part of the article reviews the main processing algorithms in the aspect of computational complexity and visual quality and their modification for applications under specific conditions. The comparison of the MM methods presented in the paper and recommendations for their applications across a wide variety of fields were supported with examples originating from recent publications. The effectiveness of visual methods based on motion magnification in machine diagnosis and the identification of malfunctions are illustrated with selected examples of the implementation derived from authors’ workshop practice under industrial conditions.

Digital image processing to detect subtle motion in stony coral

Coral reef ecosystems support significant biological activities and harbor huge diversity, but they are facing a severe crisis driven by anthropogenic activities and climate change. An important behavioral trait of the coral holobiont is coral motion, which may play an essential role in feeding, competition, reproduction, and thus survival and fitness. Therefore, characterizing coral behavior through motion analysis will aid our understanding of basic biological and physical coral functions. However, tissue motion in the stony scleractinian corals that contribute most to coral reef construction are subtle and may be imperceptible to both the human eye and commonly used imaging techniques. Here we propose and apply a systematic approach to quantify and visualize subtle coral motion across a series of light and dark cycles in the scleractinian coral Montipora capricornis. We use digital image correlation and optical flow techniques to quantify and characterize minute coral motions under different light conditions. In addition, as a visualization tool, motion magnification algorithm magnifies coral motions in different frequencies, which explicitly displays the distinctive dynamic modes of coral movement. Specifically, our assessment of displacement, strain, optical flow, and mode shape quantify coral motion under different light conditions, and they all show that M. capricornis exhibits more active motions at night compared to day. Our approach provides an unprecedented insight into micro-scale coral movement and behavior through macro-scale digital imaging, thus offering a useful empirical toolset for the coral research community.

Motion magnification analysis of microscopy videos of biological cells

It is well recognized that isolated cardiac muscle cells beat in a periodic manner. Recently, evidence indicates that other, non-muscle cells, also perform periodic motions that are either imperceptible under conventional lab microscope lens or practically not easily amenable for analysis of oscillation amplitude, frequency, phase of movement and its direction. Here, we create a real-time video analysis tool to visually magnify and explore sub-micron rhythmic movements performed by biological cells and the induced movements in their surroundings. Using this tool, we suggest that fibroblast cells perform small fluctuating movements with a dominant frequency that is dependent on their surrounding substrate and its stiffness.

Motion microscopy for label-free detection of circulating breast tumor cells

Circulating tumor cells (CTCs) are cancer cells that have been shed from a primary tumor and circulate in the bloodstream during progression of cancer. They may thus serve as circulating biomarkers that can predict, diagnose and guide therapy. Moreover, phenotypic and genotypic analysis of CTCs can facilitate prospective assessment of mutations and enable personalized treatment. A number of methodologies based on biological and physical differences between circulating tumor and non-tumor cells have been developed over the past few years. However, these methods did not have sufficient sensitivity or specificity. In this work, a remote analysis protocol was designed using motion microscopy that amplifies cellular micro motions in a captured video by re-rendering small motions to generate extreme magnified visuals to detect dynamic motions that are not otherwise visible by naked eye. Intriguingly, motion microscopy demonstrated fluctuations around breast tumor cells, which we referred to herein as cellular trail. Phenomena of cellular trail mostly emerged between 0.5 and 1.5 Hz on amplified video images. Interestingly, cellular trails were associated with cell surface proteins and flow rates rather than mitochondrial activity. Moreover, cellular trails were present only around circulating tumor cells from individuals with breast cancer under conditions of 20-30 μm/s and 0.5-1.5 Hz. Thus, motion microscopy based CTC detection method can offer a valuable supplementary diagnostic tool for assessment of drug efficacy and identifying physical characteristics of tumor cells for further research.

A Novel Approach for 3D-Structural Identification through Video Recording: Magnified Tracking

Advancements in optical imaging devices and computer vision algorithms allow the exploration of novel diagnostic techniques for use within engineering systems. A recent field of application lies in the adoption of such devices for non-contact vibrational response recordings of structures, allowing high spatial density measurements without the burden of heavy cabling associated with conventional technologies. This, however, is not a straightforward task due to the typically low-amplitude displacement response of structures under ambient operational conditions. A novel framework, namely Magnified Tracking (MT), is proposed herein to overcome this limitation through the synergistic use of two computer vision techniques. The recently proposed phase-based motion magnification (PBMM) framework, for amplifying motion in a video within a defined frequency band, is coupled with motion tracking by means of particle tracking velocimetry (PTV). An experimental campaign was conducted to validate a proof-of-concept, where the dynamic response of a shear frame was measured both by conventional sensors as well as a video camera setup, and cross-compared to prove the feasibility of the proposed non-contact approach. The methodology was explored both in 2D and 3D configurations, with PTV revealing a powerful tool for the measurement of perceptible motion. When MT is utilized for tracking “imperceptible” structural responses (i.e., below PTV sensitivity), via the use of PBMM around the resonant frequencies of the structure, the amplified motion reveals the operational deflection shapes, which are otherwise intractable. The modal results extracted from the magnified videos, using PTV, demonstrate MT to be a viable non-contact alternative for 3D modal identification with the benefit of a spatially dense measurement grid.

LED based real-time survival bioassays for nematode research

Nematode bioassays are extensively conducted worldwide, either for screening anthelmintic drugs or for assessing the toxicity of drug candidates. Recently, the US Environmental Protection Agency mandated the use of invertebrate models including nematodes especially Caenorhabditis elegans, for toxicity testing as an alternative to rodent models. The significance of nematode bioassays in the biological sciences is escalating, but no standardized protocol is available to assess nematode mortality in a liquid medium. Manual counting under white light is the only approach currently practiced, which exhibit large variabilities and false positive results. Here, we describe an innovative counting strategy that employs light-emitting diode (LED) technology. We found that the nematodes stopped moving under white light (360–760 nm) when administered with sub-lethal dosage (LC₅₀) of a toxic drug, whereas they responded rapidly to blue (450–490 nm) and ultraviolet (UV) (100–400 nm) LED lights. Furthermore, paralyzed nematodes responded in less than 5 seconds to a LED pulse. The response to the LED stimulus was distinctively noted in C. elegans dauers, which squirmed away from illuminated sites within seconds. LED produced an incoherent beam, and uniformly distributed light across the sampling area. In conclusion, this method is more accurate than the conventional counting techniques, and enables us to differentiate paralyzed and dead nematodes virtually in real-time. Furthermore, this technique would appear to be suitable for incorporating a motion-sensor based automated system.