Total variation models for variable lighting face recognition

Noise Robust Illumination Invariant Face Recognition Via Bivariate Wavelet Shrinkage in Logarithm Domain

Recognizing faces under various lighting conditions is a challenging problem in artificial intelligence and applications. In this paper we describe a new face recognition algorithm which is invariant to illumination. We first convert image files to the logarithm domain and then we implement them using the dual-tree complex wavelet transform (DTCWT) which yields images approximately invariant to changes in illumination change. We classify the images by the collaborative representation-based classifier (CRC). We also perform the following sub-band transformations: (i) we set the approximation sub-band to zero if the noise standard deviation is greater than 5; (ii) we then threshold the two highest frequency wavelet sub-bands using bivariate wavelet shrinkage. (iii) otherwise, we set these two highest frequency wavelet sub-bands to zero. On obtained images we perform the inverse DTCWT which results in illumination invariant face images. The proposed method is strongly robust to Gaussian white noise. Experimental results show that our proposed algorithm outperforms several existing methods on the Extended Yale Face Database B and the CMU-PIE face database.

A High-Performance Face Illumination Processing Method via Multi-Stage Feature Maps

In recent years, Generative Adversarial Networks (GANs)-based illumination processing of facial images has made favorable achievements. However, some GANs-based illumination-processing methods only pay attention to the image quality and neglect the recognition accuracy, whereas others only crop partial face area and ignore the challenges to synthesize photographic face, background and hair when the original face image is under extreme illumination (Image under extreme illumination (extreme illumination conditions) means that we cannot see the texture and structure information clearly and most pixel values tend to 0 or 255.). Moreover, the recognition accuracy is low when the faces are under extreme illumination conditions. For these reasons, we present an elaborately designed architecture based on convolutional neural network and GANs for processing the illumination of facial image. We use ResBlock at the down-sampling stage in our encoder and adopt skip connections in our generator. This special design together with our loss can enhance the ability to preserve identity and generate high-quality images. Moreover, we use different convolutional layers of a pre-trained feature network to extract varisized feature maps, and then use these feature maps to compute loss, which is named multi-stage feature maps (MSFM) loss. For the sake of fairly evaluating our method against state-of-the-art models, we use four metrics to estimate the performance of illumination-processing algorithms. A variety of experimental data indicate that our method is superior to the previous models under various illumination challenges in illumination processing. We conduct qualitative and quantitative experiments on two datasets, and the experimental data indicate that our scheme obviously surpasses the state-of-the-art algorithms in image quality and identification accuracy.

Improving Shadow Suppression for Illumination Robust Face Recognition

2D face analysis techniques, such as face landmarking, face recognition and face verification, are reasonably dependent on illumination conditions which are usually uncontrolled and unpredictable in the real world. The current massive data-driven approach, e.g., deep learning-based face recognition, requires a huge amount of labeled training face data that hardly cover the infinite lighting variations that can be encountered in real-life applications. An illumination robust preprocessing method thus remains a very interesting but also a significant challenge in reliable face analysis. In this paper we propose a novel model driven approach to improve lighting normalization of face images. Specifically, we propose to build the underlying reflectance model which characterizes interactions between skin surface, lighting source and camera sensor, and elaborate the formation of face color appearance. The proposed illumination processing pipeline enables generation of the Chromaticity Intrinsic Image (CII) in a log chromaticity space which is robust to illumination variations. Moreover, as an advantage over most prevailing methods, a photo-realistic color face image is subsequently reconstructed, which eliminates a wide variety of shadows whilst retaining the color information and identity details. Experimental results under different scenarios and using various face databases show the effectiveness of the proposed approach in dealing with lighting variations, including both soft and hard shadows, in face recognition.

Single Sample Face Recognition under Varying Illumination via QRCP Decomposition

In this paper, we present a novel high-frequency facial feature and a high-frequency based sparse representation classification to tackle single sample face recognition (SSFR) under varying illumination. Firstly, we propose the assumption that QRCP bases can represent intrinsic face surface features with different frequencies, and their corresponding energy coefficients describe illumination intensities. Based on this assumption, we take QRCP bases with corresponding weighting coefficients (i.e. the major components of energy coefficients) to develop the high-frequency facial feature of the face image, which is named as QRCP-face. The normalized QRCP-face (NQRCPface) is constructed to further constraint illumination effects by normalizing the weighting coefficients of QRCP-face. Moreover, we propose the adaptive QRCP-face (AQRCP-face) that assigns a special parameter to NQRCP-face via the illumination level estimated by the weighting coefficients. Secondly, we consider that the differences of pixel images cannot model the intraclass variations of generic faces with illumination variations, and the specific identification information of the generic face is redundant for the current SSFR with generic learning. To tackle above two issues, we develop a general high-frequency based sparse representation (GHSP) model. Two practical approaches separated high-frequency based sparse representation (SHSP) and unified high-frequency based sparse representation (UHSP) are developed. Finally, the performances of the proposed methods are verified on the Extended Yale B, CMU PIE, AR, LFW and our self-built Driver face databases. The experimental results indicate that the proposed methods outperform previous approaches for SSFR under varying illumination.

Region-Aware Reflection Removal with Unified Content and Gradient Priors

Removing the undesired reflections in images taken through the glass is of broad application to various image processing and computer vision tasks. Existing single image based solutions heavily rely on scene priors such as separable sparse gradients caused by different levels of blur, and they are fragile when such priors are not observed. In this paper, we notice that strong reflections usually dominant a limited region in the whole image, and propose a Region-aware Reflection Removal (R3) approach by automatically detecting and heterogeneously processing regions with and without reflections. We integrate content and gradient priors to jointly achieve missing contents restoration as well as background and reflection separation in a unified optimization framework. Extensive validation using 50 sets of real data shows that the proposed method outperforms state-of-the-art on both quantitative metrics and visual qualities.

Face recognition algorithm using extended vector quantization histogram features

In this paper, we propose a face recognition algorithm based on a combination of vector quantization (VQ) and Markov stationary features (MSF). The VQ algorithm has been shown to be an effective method for generating features; it extracts a codevector histogram as a facial feature representation for face recognition. Still, the VQ histogram features are unable to convey spatial structural information, which to some extent limits their usefulness in discrimination. To alleviate this limitation of VQ histograms, we utilize Markov stationary features (MSF) to extend the VQ histogram-based features so as to add spatial structural information. We demonstrate the effectiveness of our proposed algorithm by achieving recognition results superior to those of several state-of-the-art methods on publicly available face databases.

Full L1-regularized Traction Force Microscopy over whole cells

Background

Traction Force Microscopy (TFM) is a widespread technique to estimate the tractions that cells exert on the surrounding substrate. To recover the tractions, it is necessary to solve an inverse problem, which is ill-posed and needs regularization to make the solution stable. The typical regularization scheme is given by the minimization of a cost functional, which is divided in two terms: the error present in the data or data fidelity term; and the regularization or penalty term. The classical approach is to use zero-order Tikhonov or L₂-regularization, which uses the L₂-norm for both terms in the cost function. Recently, some studies have demonstrated an improved performance using L₁-regularization (L₁-norm in the penalty term) related to an increase in the spatial resolution and sensitivity of the recovered traction field. In this manuscript, we present a comparison between the previous two regularization schemes (relying in the L₂-norm for the data fidelity term) and the full L₁-regularization (using the L₁-norm for both terms in the cost function) for synthetic and real data.

Results

Our results reveal that L₁-regularizations give an improved spatial resolution (more important for full L₁-regularization) and a reduction in the background noise with respect to the classical zero-order Tikhonov regularization. In addition, we present an approximation, which makes feasible the recovery of cellular tractions over whole cells on typical full-size microscope images when working in the spatial domain.

Conclusions

The proposed full L₁-regularization improves the sensitivity to recover small stress footprints. Moreover, the proposed method has been validated to work on full-field microscopy images of real cells, what certainly demonstrates it is a promising tool for biological applications.

Electronic supplementary material

The online version of this article (doi:10.1186/s12859-017-1771-0) contains supplementary material, which is available to authorized users.

Lighting Equilibrium Distribution Maps and Their Application to Face Recognition Under Difficult Lighting Conditions

In this paper, a novel facial-patch based recognition framework is proposed to deal with the problem of face recognition (FR) on the serious illumination condition. First, a novel lighting equilibrium distribution maps (LEDM) for illumination normalization is proposed. In LEDM, an image is analyzed in logarithm domain with wavelet transform, and the approximation coefficients of the image are mapped according to a reference-illumination map in order to normalize the distribution of illumination energy due to different lighting effects. Meanwhile, the detail coefficients are enhanced to achieve detail information emphasis. The LEDM is obtained by blurring the distances between the test image and the reference illumination map in the logarithm domain, which may express the entire distribution of illumination variations. Then, a facial-patch based framework and a credit degree based facial patches synthesizing algorithm are proposed. Each normalized face images is divided into several stacked patches. And, all patches are individually classified, then each patch from the test image casts a vote toward the parent image classification. A novel credit degree map is established based on the LEDM, which is deciding a credit degree for each facial patch. The main idea of credit degree map construction is the over-and under-illuminated regions should be assigned lower credit degree than well-illuminated regions. Finally, results are obtained by the credit degree based facial patches synthesizing. The proposed method provides state-of-the-art performance on three data sets that are widely used for testing FR under different illumination conditions: Extended Yale-B, CAS-PEAL-R1, and CMUPIE. Experimental results show that our FR frame outperforms several existing illumination compensation methods.

Enhanced Local Gradient Order Features and Discriminant Analysis for Face Recognition

Robust descriptor-based subspace learning with complex data is an active topic in pattern analysis and machine intelligence. A few researches concentrate the optimal design on feature representation and metric learning. However, traditionally used features of single-type, e.g., image gradient orientations (IGOs), are deficient to characterize the complete variations in robust and discriminant subspace learning. Meanwhile, discontinuity in edge alignment and feature match are not been carefully treated in the literature. In this paper, local order constrained IGOs are exploited to generate robust features. As the difference-based filters explicitly consider the local contrasts within neighboring pixel points, the proposed features enhance the local textures and the order-based coding ability, thus discover intrinsic structure of facial images further. The multimodal features are automatically fused in the most discriminant subspace. The utilization of adaptive interaction function suppresses outliers in each dimension for robust similarity measurement and discriminant analysis. The sparsity-driven regression model is modified to adapt the classification issue of the compact feature representation. Extensive experiments are conducted by using some benchmark face data sets, e.g., of controlled and uncontrolled environments, to evaluate our new algorithm.

Light Field Imaging Based Accurate Image Specular Highlight Removal

Specular reflection removal is indispensable to many computer vision tasks. However, most existing methods fail or degrade in complex real scenarios for their individual drawbacks. Benefiting from the light field imaging technology, this paper proposes a novel and accurate approach to remove specularity and improve image quality. We first capture images with specularity by the light field camera (Lytro ILLUM). After accurately estimating the image depth, a simple and concise threshold strategy is adopted to cluster the specular pixels into “unsaturated” and “saturated” category. Finally, a color variance analysis of multiple views and a local color refinement are individually conducted on the two categories to recover diffuse color information. Experimental evaluation by comparison with existed methods based on our light field dataset together with Stanford light field archive verifies the effectiveness of our proposed algorithm.

Compositional Dictionaries for Domain Adaptive Face Recognition

We present a dictionary learning approach to compensate for the transformation of faces due to the changes in view point, illumination, resolution, and so on. The key idea of our approach is to force domain-invariant sparse coding, i.e., designing a consistent sparse representation of the same face in different domains. In this way, the classifiers trained on the sparse codes in the source domain consisting of frontal faces can be applied to the target domain (consisting of faces in different poses, illumination conditions, and so on) without much loss in recognition accuracy. The approach is to first learn a domain base dictionary, and then describe each domain shift (identity, pose, and illumination) using a sparse representation over the base dictionary. The dictionary adapted to each domain is expressed as the sparse linear combinations of the base dictionary. In the context of face recognition, with the proposed compositional dictionary approach, a face image can be decomposed into sparse representations for a given subject, pose, and illumination. This approach has three advantages. First, the extracted sparse representation for a subject is consistent across domains, and enables pose and illumination insensitive face recognition. Second, sparse representations for pose and illumination can be subsequently used to estimate the pose and illumination condition of a face image. Last, by composing sparse representations for the subject and the different domains, we can also perform pose alignment and illumination normalization. Extensive experiments using two public face data sets are presented to demonstrate the effectiveness of the proposed approach for face recognition.

Discriminative and Compact Coding for Robust Face Recognition

In this paper, we propose a novel discriminative and compact coding (DCC) for robust face recognition. It introduces multiple error measurements into regression model. They collaborate to tune regression codes of different properties (sparsity, compactness, high discriminating ability, etc.), to further improve robustness and adaptivity of the regression model. We propose two types of coding models: 1) multiscale error measurements that produces sparse and highly discriminative codes and 2) inspires within-class collaborative representation that produces sparse and compact codes. The update of codes and the combination of different errors are automatically processed. DCC is also robust to the choice of parameters, producing stable regression residuals which are crucial to classification. Extensive experiments on benchmark datasets show that DCC has promising performance and outperforms other state-of-the-art regression models.

Face liveness detection from a single image via diffusion speed model

Spoofing using photographs or videos is one of the most common methods of attacking face recognition and verification systems. In this paper, we propose a real-time and nonintrusive method based on the diffusion speed of a single image to address this problem. In particular, inspired by the observation that the difference in surface properties between a live face and a fake one is efficiently revealed in the diffusion speed, we exploit antispoofing features by utilizing the total variation flow scheme. More specifically, we propose defining the local patterns of the diffusion speed, the so-called local speed patterns, as our features, which are input into the linear SVM classifier to determine whether the given face is fake or not. One important advantage of the proposed method is that, in contrast to previous approaches, it accurately identifies diverse malicious attacks regardless of the medium of the image, e.g., paper or screen. Moreover, the proposed method does not require any specific user action. Experimental results on various data sets show that the proposed method is effective for face liveness detection as compared with previous approaches proposed in studies in the literature.

Multiscale logarithm difference edgemaps for face recognition against varying lighting conditions

Lambertian model is a classical illumination model consisting of a surface albedo component and a light intensity component. Some previous researches assume that the light intensity component mainly lies in the large-scale features. They adopt holistic image decompositions to separate it out, but it is difficult to decide the separating point between large-scale and small-scale features. In this paper, we propose to take a logarithm transform, which can change the multiplication of surface albedo and light intensity into an additive model. Then, a difference (substraction) between two pixels in a neighborhood can eliminate most of the light intensity component. By dividing a neighborhood into subregions, edgemaps of multiple scales can be obtained. Then, each edgemap is multiplied by a weight that can be determined by an independent training scheme. Finally, all the weighted edgemaps are combined to form a robust holistic feature map. Extensive experiments on four benchmark data sets in controlled and uncontrolled lighting conditions show that the proposed method has promising results, especially in uncontrolled lighting conditions, even mixed with other complicated variations.

Multilayer surface albedo for face recognition with reference images in bad lighting conditions

In this paper, we propose a multilayer surface albedo (MLSA) model to tackle face recognition in bad lighting conditions, especially with reference images in bad lighting conditions. Some previous researches conclude that illumination variations mainly lie in the large-scale features of an image and extract small-scale features in the surface albedo (or surface texture). However, this surface albedo is not robust enough, which still contains some detrimental sharp features. To improve robustness of the surface albedo, MLSA further decomposes it as a linear sum of several detailed layers, to separate and represent features of different scales in a more specific way. Then, the layers are adjusted by separate weights, which are global parameters and selected for only once. A criterion function is developed to select these layer weights with an independent training set. Despite controlled illumination variations, MLSA is also effective to uncontrolled illumination variations, even mixed with other complicated variations (expression, pose, occlusion, and so on). Extensive experiments on four benchmark data sets show that MLSA has good receiver operating characteristic curve and statistical discriminating capability. The refined albedo improves recognition performance, especially with reference images in bad lighting conditions.

Decomposition and extraction: a new framework for visual classification

In this paper, we present a novel framework for visual classification based on hierarchical image decomposition and hybrid midlevel feature extraction. Unlike most midlevel feature learning methods, which focus on the process of coding or pooling, we emphasize that the mechanism of image composition also strongly influences the feature extraction. To effectively explore the image content for the feature extraction, we model a multiplicity feature representation mechanism through meaningful hierarchical image decomposition followed by a fusion step. In particularly, we first propose a new hierarchical image decomposition approach in which each image is decomposed into a series of hierarchical semantical components, i.e, the structure and texture images. Then, different feature extraction schemes can be adopted to match the decomposed structure and texture processes in a dissociative manner. Here, two schemes are explored to produce property related feature representations. One is based on a single-stage network over hand-crafted features and the other is based on a multistage network, which can learn features from raw pixels automatically. Finally, those multiple midlevel features are incorporated by solving a multiple kernel learning task. Extensive experiments are conducted on several challenging data sets for visual classification, and experimental results demonstrate the effectiveness of the proposed method.

ILLUMINATION PROCESSING IN FACE RECOGNITION

Changes in light intensity and angle present a major challenge to the creation of reliable face recognition systems. The existence of bright regions and dark regions has been shown to have a serious negative impact on the performance of face recognition systems. This paper proposes a solution to this problem based on self-quotient image (SQI) processing method. In this method, bright and dark areas are processed separately without changing the essential characteristics of the image of the face. The dark and light areas are processed separately by SQI. Experimental results indicate that this Single-Light-Region and Single-Dark-Region SQI method removes the adverse effect of multi-bright and multi-dark areas better than competing methods.

Texture enhanced histogram equalization using TV- L¹ image decomposition

Histogram transformation defines a class of image processing operations that are widely applied in the implementation of data normalization algorithms. In this paper, we present a new variational approach for image enhancement that is constructed to alleviate the intensity saturation effects that are introduced by standard contrast enhancement (CE) methods based on histogram equalization. In this paper, we initially apply total variation (TV) minimization with a L(1) fidelity term to decompose the input image with respect to cartoon and texture components. Contrary to previous papers that rely solely on the information encompassed in the distribution of the intensity information, in this paper, the texture information is also employed to emphasize the contribution of the local textural features in the CE process. This is achieved by implementing a nonlinear histogram warping CE strategy that is able to maximize the information content in the transformed image. Our experimental study addresses the CE of a wide variety of image data and comparative evaluations are provided to illustrate that our method produces better results than conventional CE strategies.

Robust Face Recognition Based on Texture Analysis

In this paper, we present a new framework for face recognition with varying illumination based on DCT total variation minimization (DTV), a Gabor filter, a sub-micro-pattern analysis (SMP) and discriminated accumulative feature transform (DAFT). We first suppress the illumination effect by using the DCT with the help of TV as a tool for face normalization. The DTV image is then emphasized by the Gabor filter. The facial features are encoded by our proposed method - the SMP. The SMP image is then transformed to the 2D histogram using DAFT. Our system is verified with experiments on the AR and the Yale face database B.

MULTI-SCALE GRADIENT INVARIANT FOR FACE RECOGNITION UNDER VARYING ILLUMINATION

In this paper, a novel approach derived from image gradient domain called multi-scale gradient faces (MGF) is proposed to abstract multi-scale illumination-insensitive measure for face recognition. MGF applies multi-scale analysis on image gradient information, which can discover underlying inherent structure in images and keep the details at most while removing varying lighting. The proposed approach provides state-of-the-art performance on Extended YaleB and PIE: Recognition rates of 99.11% achieved on PIE database and 99.38% achieved on YaleB which outperforms most existing approaches. Furthermore, the experimental results on noised Yale-B validate that MGF is more robust to image noise.

Local Relation Map: A Novel Illumination Invariant Face Recognition Approach

In this paper, a novel illumination invariant face recognition approach is proposed. Different from most existing methods, an additive term as noise is considered in the face model under varying illuminations in addition to a multiplicative illumination term. High frequency coefficients of Discrete Cosine Transform (DCT) are discarded to eliminate the effect caused by noise. Based on the local characteristics of the human face, a simple but effective illumination invariant feature local relation map is proposed. Experimental results on the Yale B, Extended Yale B and CMU PIE demonstrate the outperformance and lower computational burden of the proposed method compared to other existing methods. The results also demonstrate the validity of the proposed face model and the assumption on noise.

Illumination compensation using oriented local histogram equalization and its application to face recognition

Illumination compensation and normalization play a crucial role in face recognition. The existing algorithms either compensated low-frequency illumination, or captured high-frequency edges. However, the orientations of edges were not well exploited. In this paper, we propose the orientated local histogram equalization (OLHE) in brief, which compensates illumination while encoding rich information on the edge orientations. We claim that edge orientation is useful for face recognition. Three OLHE feature combination schemes were proposed for face recognition: 1) encoded most edge orientations; 2) more compact with good edge-preserving capability; and 3) performed exceptionally well when extreme lighting conditions occurred. The proposed algorithm yielded state-of-the-art performance on AR, CMU PIE, and extended Yale B using standard protocols. We further evaluated the average performance of the proposed algorithm when the images lighted differently were observed, and the proposed algorithm yielded the promising results.

Face distortion aftereffects in personally familiar, famous, and unfamiliar faces

The internal face prototype is thought to be a construction of the average of every previously viewed face (Schwaninger et al., 2003). However, the influence of the most frequently encountered faces (i.e., personally familiar faces) has been generally understated. The current research explored the face distortion aftereffect in unfamiliar, famous, and personally familiar (each subject’s parent) faces. Forty-eight adult participants reported whether faces were distorted or not (distorted by shifting the eyes in the vertical axis) of a series of images that included unfamiliar, famous, and personally familiar faces. The number of faces perceived to be “odd” was measured pre- and post-adaptation to the most extreme distortion. Participants were adapted to either an unfamiliar, famous, or personally familiar face. The results indicate that adaptation transferred from unfamiliar faces to personally familiar faces more so than the converse and aftereffects did not transfer from famous faces to unfamiliar faces. These results are indicative of representation differences between unfamiliar, famous, and personally familiar faces, whereby personally familiar faces share representations of both unfamiliar and famous faces.