In vivo panoramic human skin shape and deformation measurement using mirror-assisted multi-view digital image correlation

Stereo Camera Setup for 360° Digital Image Correlation to Reveal Smart Structures of Hakea Fruits

About forty years after its first application, digital image correlation (DIC) has become an established method for measuring surface displacements and deformations of objects under stress. To date, DIC has been used in a variety of in vitro and in vivo studies to biomechanically characterise biological samples in order to reveal biomimetic principles. However, when surfaces of samples strongly deform or twist, they cannot be thoroughly traced. To overcome this challenge, different DIC setups have been developed to provide additional sensor perspectives and, thus, capture larger parts of an object's surface. Herein, we discuss current solutions for this multi-perspective DIC, and we present our own approach to a 360° DIC system based on a single stereo-camera setup. Using this setup, we are able to characterise the desiccation-driven opening mechanism of two woody Hakea fruits over their entire surfaces. Both the breaking mechanism and the actuation of the two valves in predominantly dead plant material are models for smart materials. Based on these results, an evaluation of the setup for 360° DIC regarding its use in deducing biomimetic principles is given. Furthermore, we propose a way to improve and apply the method for future measurements.

Panoramic three-dimensional optical digitization system assisted by a bi-mirror

The digitization of objects' full surfaces finds widespread applications in fields such as virtual reality, art and design, and medical and biological sciences. For the realization of three-dimensional full-surface digitization of objects within complex sceneries, we propose a straightforward, efficient, and robust panoramic three-dimensional optical digitization system. This system contains a laser-based optical three-dimensional measurement system and a bi-mirror. By integrating mirrors into the system, we enable the illumination of the object from all angles using the projected laser beam in a single scanning process. Moreover, the main camera employed in the system can acquire three-dimensional information of the object from several different viewpoints. The rotational scanning method enhances the efficiency and applicability of the three-dimensional scanning process, enabling the acquisition of surface information of large-scale objects. After obtaining the three-dimensional data of the sample from different viewpoints using laser triangulation, mirror reflection transformation was employed to obtain the full-surface three-dimensional data of the object in the global coordinate system. The proposed method has been subjected to precision and validity experiments using samples with different surface characteristics and sizes, resulting in the demonstration of its capability for achieving correct three-dimensional digitization of the entire surface in diverse complex sceneries.

Occlusion-robust scene flow-based tissue deformation recovery incorporating a mesh optimization model

PURPOSE

Tissue deformation recovery is to reconstruct the change in shape and surface strain caused by tool-tissue interaction or respiration, which is essential for providing motion and shape information that benefits the improvement of the safety of minimally invasive surgery. The binocular vision-based approach is a practical candidate for deformation recovery as no extra devices are required. However, previous methods suffer from limitations such as the reliance on biomechanical priors and the vulnerability to the occlusion caused by surgical instruments. To address the issues, we propose a deformation recovery method incorporating mesh structures and scene flow.

METHODS

The method can be divided into three modules. The first one is the implementation of the two-step scene flow generation module to extract the 3D motion from the binocular sequence. Second, we propose a strain-based filtering method to denoise the original scene flow. Third, a mesh optimization model is proposed that strengthens the robustness to occlusion by employing contextual connectivity.

RESULTS

In a phantom and an in vivo experiment, the feasibility of the method in recovering surface deformation in the presence of tool-induced occlusion was demonstrated. Surface reconstruction accuracy was quantitatively evaluated by comparing the recovered mesh surface with the 3D scanned model in the phantom experiment. Results show that the overall error is 0.70 ± 0.55 mm.

CONCLUSION

The method has been demonstrated to be capable of continuously recovering surface deformation using mesh representation with robustness to the occlusion caused by surgical forceps and promises to be suitable for the application in actual surgery.

Identifiability of soft tissue constitutive parameters from in-vivo macro-indentation

Reliable identification of soft tissue material parameters is frequently required in a variety of applications, particularly for biomechanical simulations using finite element analysis (FEA). However, determining representative constitutive laws and material parameters is challenging and often comprises a bottleneck that hinders the successful implementation of FEA. Soft tissues exhibit a nonlinear response and are commonly modeled using hyperelastic constitutive laws. In-vivo material parameter identification, for which standard mechanical tests (e.g., uniaxial tension and compression) are inapplicable, is commonly achieved using finite macro-indentation test. Due to the lack of analytical solutions, the parameters are commonly identified using inverse FEA (iFEA), in which simulated results and experimental data are iteratively compared. However, determining what data must be collected to accurately identify a unique parameter set remains unclear. This work investigates the sensitivities of two types of measurements: indentation force-depth data (e.g., measured using an instrumented indenter) and full-field surface displacements (e.g., using digital image correlation). To eliminate model fidelity and measurement-related errors, we employed an axisymmetric indentation FE model to produce synthetic data for four 2-parameter hyperelastic constitutive laws: compressible Neo-Hookean, and nearly incompressible Mooney-Rivlin, Ogden, and Ogden-Moerman models. For each constitutive law, we computed the objective functions representing the discrepancies in the reaction force, the surface displacement, and their combination, and visualized them for hundreds of parameter sets, spanning a representative range as found in the literature for the bulk soft tissue complex in human lower limbs. Moreover, we quantified three identifiability metrics, which provided insights into the uniqueness (or lack thereof) and the sensitivities. This approach provides a clear and systematic evaluation of the parameter identifiability, which is independent of the selection of the optimization algorithm and initial guesses required in iFEA. Our analysis indicated that the indenter's force-depth data, despite being commonly used for parameter identification, was insufficient for reliably and accurately identifying both parameters for all the investigated material models and that the surface displacement data improved the parameter identifiability in all cases, although the Mooney-Rivlin parameters remained poorly identifiable. Informed by the results, we then discuss several identification strategies for each constitutive model. Finally, we openly provide the codes used in this study, to allow others to further investigate the indentation problem according to their specifications (e.g., by modifying the geometries, dimensions, mesh, material models, boundary conditions, contact parameters, or objective functions).

Panoramic/Dual-Surface Digital Image Correlation Measurement Using a Single Camera

We propose a cost-effective and simple-to-implement mirror-assisted single-camera panoramic digital image correlation (DIC) method for panoramic/dual-surface profile and deformation measurement. Specifically, two planar mirrors and a single camera attached with a four-mirror adapter are used to capture stereo images of the front and rear surfaces of a test object. These stereo images can be processed by regular stereo-DIC to retrieve shape and kinematics fields of each surface. Further, with the speckle patterns prefabricated on the mirrors, reflection transformation matrices are obtained and applied to transform all reconstructed surfaces into a common world coordinate system. As such, panoramic/dual-surface shape and deformation measurements can be realized. For validation, a high-resolution smartphone camera and an industrial camera were, respectively, used to construct mirror-assisted single-camera panoramic DIC systems. Real experiments, including panoramic shape measurement of an aluminum cylinder, dual-surface shape measurement of an aluminum plate and uniaxial tensile tests of aluminum sheet specimens, were performed, confirming the feasibility and accuracy of the method. Since only a single camera and a few auxiliary reflective mirrors are required, the proposed method provides a cost-effective and convenient way for taking panoramic/dual-surface shape and deformation measurements of regular-sized cylindrical and bar samples.

Application of Digital Image Correlation in Structural Health Monitoring of Bridge Infrastructures: A Review

A vision-based approach has been employed in Structural Health Monitoring (SHM) of bridge infrastructure. The approach has many advantages: non-contact, non-destructive, long-distance, high precision, immunity from electromagnetic interference, and multiple-target monitoring. This review aims to summarise the vision- and Digital Image Correlation (DIC)-based SHM methods for bridge infrastructure because of their strategic significance and security concerns. Four different bridge types were studied: concrete, suspension, masonry, and steel bridge. DIC applications in SHM have recently garnered attention in aiding to assess the bridges’ structural response mechanisms under loading. Different non-destructive diagnostics methods for SHM in civil infrastructure have been used; however, vision-based techniques like DIC were only developed over the last two decades, intending to facilitate damage detection in bridge systems with prompt and accurate data for efficient and sustainable operation of the bridge structure throughout its service life. Research works reviewed in this article demonstrated the DIC capability to detect damage such as cracks, spalling, and structural parameters such as deformation, strains, vibration, deflection, and rotation. In addition, the reviewed works indicated that the DIC as an efficient and reliable technique could provide sustainable monitoring solutions for different bridge infrastructures.

Full-view imaging on dynamic closed surface by curved-to-flat conversion lens

Conventional full-view imaging systems, which often need complicated image processing algorithms to reconstruct full-view images captured by motional/multiple cameras from different views, cannot have good real-time imaging capability. We design curved-to-flat conversion lens (CFCL) based on optic-null medium, which can directly project/image optical patterns from closed object surface onto image plane (e.g., the focal plane of microscopy), and shows good real-time full-view imaging performance. To realize the CFCL, the reduced optic-null medium is designed by subwavelength metal channels filled with homogeneous isotropic dielectrics. Numerical simulation results verify the function of the designed CFCL, which can image various dynamic optical patterns from the closed object surface to the finite-view image plane. The designed CFCL may have many applications in real-timely observing dynamic closed surfaces in full view, e.g., living tissue/cell and soft material's surface.

Calibrating mirror-assisted multi-view digital image correlation system using a speckled planar object

Recently, a novel low-cost and easy-to-implement mirror-assisted multi-view digital image correlation (MV-DIC) technique that uses only a vertically arranged binocular stereo-digital image correlation (stereo-DIC) system and two planar mirrors was proposed for panoramic/dual-surface kinematic field measurements. The mirror-assisted MV-DIC, which is equivalent to a regular MV-DIC system comprising three/two identical stereo-DIC systems, can simultaneously measure three/two different regions of a test sample. To perform panoramic/dual-surface kinematic field measurements with this technique, one of the most important procedures is accurately calibrating the reflection transformation of each mirror to transform all measured regions to a common world coordinate system. Previous works calibrate the reflection transformation by decorating speckle patterns on the mirrors or adjusting the mirrors to various poses, which either occupies precious spatial resolution of the sensor or is complicated in implementation. In this work, an easy-to-implement and practical method that employs only a speckled planar object is proposed to calibrate the reflection transformation of the planar mirrors. The effectiveness and accuracy of the proposed method are well evaluated and validated. Compared with existing methods, the proposed method holds the advantage of being efficient in sensor utilization and simple in implementation and is therefore highly recommended for the practical use of the mirror-assisted MV-DIC technique.