Masked Face Recognition Using Deep Learning: A Review

Joint Masked Face Recognition and Temperature Measurement System Using Convolutional Neural Networks

With the outbreak of COVID-19, epidemic prevention has become a way to prevent the spread of epidemics. Many public places, such as hospitals, schools, and office places, require disinfection and temperature measurement. To implement epidemic prevention systems and reduce the risk of infection, it is a recent trend to measure body temperature through non-contact sensing systems with thermal imaging cameras. Compared to fingerprints and irises, face recognition is accurate and does not require close contact, which significantly reduces the risk of infection. However, masks block most facial features, resulting in the low accuracy of face recognition systems. This work combines masked face recognition with a thermal imaging camera for use as an automated attendance system. It can record body temperature and recognize the person at the same time. Through the designed UI system, we can search the attendance information of each person. We not only provide the design method based on convolutional neural networks (CNNs), but also provide the complete embedded system as a real demonstration and achieve a 94.1% accuracy rate of masked face recognition in the real world. With the face recognition system combined with a thermal imaging camera, the purpose of screening body temperature when checking in at work can be achieved.

A survey on computer vision based human analysis in the COVID-19 era

The emergence of COVID-19 has had a global and profound impact, not only on society as a whole, but also on the lives of individuals. Various prevention measures were introduced around the world to limit the transmission of the disease, including face masks, mandates for social distancing and regular disinfection in public spaces, and the use of screening applications. These developments also triggered the need for novel and improved computer vision techniques capable of ( i ) providing support to the prevention measures through an automated analysis of visual data, on the one hand, and ( ii ) facilitating normal operation of existing vision-based services, such as biometric authentication schemes, on the other. Especially important here, are computer vision techniques that focus on the analysis of people and faces in visual data and have been affected the most by the partial occlusions introduced by the mandates for facial masks. Such computer vision based human analysis techniques include face and face-mask detection approaches, face recognition techniques, crowd counting solutions, age and expression estimation procedures, models for detecting face-hand interactions and many others, and have seen considerable attention over recent years. The goal of this survey is to provide an introduction to the problems induced by COVID-19 into such research and to present a comprehensive review of the work done in the computer vision based human analysis field. Particular attention is paid to the impact of facial masks on the performance of various methods and recent solutions to mitigate this problem. Additionally, a detailed review of existing datasets useful for the development and evaluation of methods for COVID-19 related applications is also provided. Finally, to help advance the field further, a discussion on the main open challenges and future research direction is given at the end of the survey. This work is intended to have a broad appeal and be useful not only for computer vision researchers but also the general public.

One shot learning approach for cross spectrum periocular verification

The use of face mask during the COVID-19 pandemic has increased the popularity of the periocular biometrics in surveillance applications. Despite of the rapid advancements in this area, matching images over cross spectrum is still a challenging problem. Reason may be two-fold 1) variations in image illumination 2) small size of available data sets and/or class imbalance problem. This paper proposes Siamese architecture based convolutional neural networks which works on the concept of one-shot classification. In one shot classification, network requires a single training example from each class to train the complete model which may lead to reduce the need of large dataset as well as doesn't matter whether the dataset is imbalance. The proposed architectures comprise of identical subnetworks with shared weights whose performance is assessed on three publicly available databases namely IMP, UTIRIS and PolyU with four different loss functions namely Binary cross entropy loss, Hinge loss, contrastive loss and Triplet loss. In order to mitigate the inherent illumination variations of cross spectrum images CLAHE was used to preprocess images. Extensive experiments show that the proposed Siamese CNN model with triplet loss function outperforms the states of the art periocular verification methods for cross, mono and multi spectral periocular image matching.

A Two-Stage Deep Generative Model for Masked Face Synthesis

Research on face recognition with masked faces has been increasingly important due to the prolonged COVID-19 pandemic. To make face recognition practical and robust, a large amount of face image data should be acquired for training purposes. However, it is difficult to obtain masked face images for each human subject. To cope with this difficulty, this paper proposes a simple yet practical method to synthesize a realistic masked face for an unseen face image. For this, a cascade of two convolutional auto-encoders (CAEs) has been designed. The former CAE generates a pose-alike face wearing a mask pattern, which is expected to fit the input face in terms of pose view. The output of the former CAE is readily fed into the secondary CAE for extracting a segmentation map that localizes the mask region on the face. Using the segmentation map, the mask pattern can be successfully fused with the input face by means of simple image processing techniques. The proposed method relies on face appearance reconstruction without any facial landmark detection or localization techniques. Extensive experiments with the GTAV Face database and Labeled Faces in the Wild (LFW) database show that the two complementary generators could rapidly and accurately produce synthetic faces even for challenging input faces (e.g., low-resolution face of 25 × 25 pixels with out-of-plane rotations).

Hyperspectral Face Recognition with Adaptive and Parallel SVMs in Partially Hidden Face Scenarios

Hyperspectral imaging opens up new opportunities for masked face recognition via discrimination of the spectral information obtained by hyperspectral sensors. In this work, we present a novel algorithm to extract facial spectral-features from different regions of interests by performing computer vision techniques over the hyperspectral images, particularly Histogram of Oriented Gradients. We have applied this algorithm over the UWA-HSFD dataset to extract the facial spectral-features and then a set of parallel Support Vector Machines with custom kernels, based on the cosine similarity and Euclidean distance, have been trained on fly to classify unknown subjects/faces according to the distance of the visible facial spectral-features, i.e., the regions that are not concealed by a face mask or scarf. The results draw up an optimal trade-off between recognition accuracy and compression ratio in accordance with the facial regions that are not occluded.

Masked face recognition with principal random forest convolutional neural network (PRFCNN)

Masked face recognition embarks the interest among the researchers to find a better algorithm to improve the performance of face recognition applications, especially in the Covid-19 pandemic lately. This paper introduces a proposed masked face recognition method known as Principal Random Forest Convolutional Neural Network (PRFCNN). This method utilizes the strengths of Principal Component Analysis (PCA) with the combination of Random Forest algorithm in Convolution Neural Network to pre-train the masked face features. PRFCNN is designed to assist in extracting more salient features and prevent overfitting problems. Experiments are conducted on two benchmarked datasets, RMFD (Real-World Masked Face Dataset) and LFW Simulated Masked Face Dataset using various parameter settings. The experimental result with a minimum recognition rate of 90% accuracy promises the effectiveness of the proposed PRFCNN over the other state-of-the-art methods.

A survey of uncover misleading and cyberbullying on social media for public health

Misleading health information is a critical phenomenon in our modern life due to advance in technology. In fact, social media facilitated the dissemination of information, and as a result, misinformation spread rapidly, cheaply, and successfully. Fake health information can have a significant effect on human behavior and attitudes. This survey presents the current works developed for misleading information detection (MLID) in health fields based on machine learning and deep learning techniques and introduces a detailed discussion of the main phases of the generic adopted approach for MLID. In addition, we highlight the benchmarking datasets and the most used metrics to evaluate the performance of MLID algorithms are discussed and finally, a deep investigation of the limitations and drawbacks of the current progressing technologies in various research directions is provided to help the researchers to use the most proper methods in this emerging task of MLID.

VGG16-random fourier hybrid model for masked face recognition

With the recent COVID-19 pandemic, wearing masks has become a necessity in our daily lives. People are encouraged to wear masks to protect themselves from the outside world and thus from infection with COVID-19. The presence of masks raised serious concerns about the accuracy of existing facial recognition systems since most of the facial features are obscured by the mask. To address these challenges, a new method for masked face recognition is proposed that combines a cropping-based approach (upper half of the face) with an improved VGG-16 architecture. The finest features from the un-occluded facial region are extracted using a transfer learned VGG-16 model (Forehead and eyes). The optimal cropping ratio is investigated to give an enhanced feature representation for recognition. To avoid the overhead of bias, the obtained feature vector is mapped into a lower-dimensional feature representation using a Random Fourier Feature extraction module. Comprehensive experiments on the Georgia Tech Face Dataset, Head Pose Image Dataset, and Face Dataset by Robotics Lab show that the proposed approach outperforms other state-of-the-art approaches for masked face recognition.

Effective Attention-Based Mechanism for Masked Face Recognition

Research on facial recognition has recently been flourishing, which has led to the introduction of many robust methods. However, since the worldwide outbreak of COVID-19, people have had to regularly wear facial masks, thus making existing face recognition methods less reliable. Although normal face recognition methods are nearly complete, masked face recognition (MFR)—which refers to recognizing the identity of an individual when people wear a facial mask—remains the most challenging topic in this area. To overcome the difficulties involved in MFR, a novel deep learning method based on the convolutional block attention module (CBAM) and angular margin ArcFace loss is proposed. In the method, CBAM is integrated with convolutional neural networks (CNNs) to extract the input image feature maps, particularly of the region around the eyes. Meanwhile, ArcFace is used as a training loss function to optimize the feature embedding and enhance the discriminative feature for MFR. Because of the insufficient availability of masked face images for model training, this study used the data augmentation method to generate masked face images from a common face recognition dataset. The proposed method was evaluated using the well-known masked image version of LFW, AgeDB-30, CFP-FP, and real mask image MFR2 verification datasets. A variety of experiments confirmed that the proposed method offers improvements for MFR compared to the current state-of-the-art methods.