Accurate depth estimation of skin surface using a light-field camera toward dynamic haptic palpation

Skin tactile surface restoration using deep learning from a mobile image: An application for virtual skincare

BACKGROUND

For virtual skincare using a touch feedback interface, reconstructing a 3D skin tactile surface from a mobile skin image is imperative for a dermatologist to palpate the skin surface that presents tactile characteristics of the subcutaneous tissues. However, the precise tactile reconstruction from a single view image is a challenging research problem due to varying illumination conditions.

METHODS

In this study, a deep learning-based tactile reconstruction scheme is proposed to restore tactile properties from light distortion and reconstruct the 3D tactile surface from a mobile skin image. Our method consists of light distortion removal using deep learning, cGAN, and 3D tactile surface generation based on image gradients.

RESULTS

The proposed method was tested by conducting two evaluation experiments in terms of removing light distortion and reconstructing 3D skin tactile surface in comparison with other well-known methods. The results demonstrated that our method outperforms existing other methods in both illumination-free image restoration and 3D surface reconstruction.

CONCLUSION

The proposed method is a promising approach in that tactile property distorted by illuminations can be completely restored using deep learning with a smaller training set and the precise reconstruction of 3D skin tactile surface can be achieved to be ready for a remotely touchable interface for virtual skincare applications.

Adaptive Fine Distortion Correction Method for Stereo Images of Skin Acquired with a Mobile Phone

With the development of the mobile phone, we can acquire high-resolution images of the skin to observe its detailed features using a mobile camera. We acquire stereo images using a mobile camera to enable a three-dimensional (3D) analysis of the skin surface. However, geometric changes in the observed skin structure caused by the lens distortion of the mobile phone result in a low accuracy of the 3D information extracted through stereo matching. Therefore, our study proposes a Distortion Correction Matrix (DCM) to correct the fine distortion of close-up mobile images, pixel by pixel. We verified the correction performance by analyzing the results of correspondence point matching in the stereo image corrected using the DCM. We also confirmed the correction results of the image taken at the five different working distances and derived a linear regression model for the relationship between the angle of the image and the distortion ratio. The proposed DCM considers the distortion degree, which appears to be different in the left and right regions of the image. Finally, we performed a fine distortion correction, which is difficult to check with the naked eye. The results of this study can enable the accurate and precise 3D analysis of the skin surface using corrected mobile images.