Facial Micro-Expression Recognition Based on Deep Local-Holistic Network

MCCA-VNet: A Vit-Based Deep Learning Approach for Micro-Expression Recognition Based on Facial Coding

In terms of facial expressions, micro-expressions are more realistic than macro-expressions and provide more valuable information, which can be widely used in psychological counseling and clinical diagnosis. In the past few years, deep learning methods based on optical flow and Transformer have achieved excellent results in this field, but most of the current algorithms are mainly concentrated on establishing a serialized token through the self-attention model, and they do not take into account the spatial relationship between facial landmarks. For the locality and changes in the micro-facial conditions themselves, we propose the deep learning model MCCA-VNET on the basis of Transformer. We effectively extract the changing features as the input of the model, fusing channel attention and spatial attention into Vision Transformer to capture correlations between features in different dimensions, which enhances the accuracy of the identification of micro-expressions. In order to verify the effectiveness of the algorithm mentioned, we conduct experimental testing in the SAMM, CAS (ME) II, and SMIC datasets and compared the results with other former best algorithms. Our algorithms can improve the UF1 score and UAR score to, respectively, 0.8676 and 0.8622 for the composite dataset, and they are better than other algorithms on multiple indicators, achieving the best comprehensive performance.

Micro-expression recognition based on multi-scale 3D residual convolutional neural network

In demanding application scenarios such as clinical psychotherapy and criminal interrogation, the accurate recognition of micro-expressions is of utmost importance but poses significant challenges. One of the main difficulties lies in effectively capturing weak and fleeting facial features and improving recognition performance. To address this fundamental issue, this paper proposed a novel architecture based on a multi-scale 3D residual convolutional neural network. The algorithm leveraged a deep 3D-ResNet50 as the skeleton model and utilized the micro-expression optical flow feature map as the input for the network model. Drawing upon the complex spatial and temporal features inherent in micro-expressions, the network incorporated multi-scale convolutional modules of varying sizes to integrate both global and local information. Furthermore, an attention mechanism feature fusion module was introduced to enhance the model's contextual awareness. Finally, to optimize the model's prediction of the optimal solution, a discriminative network structure with multiple output channels was constructed. The algorithm's performance was evaluated using the public datasets SMIC, SAMM, and CASME Ⅱ. The experimental results demonstrated that the proposed algorithm achieves recognition accuracies of 74.6, 84.77 and 91.35% on these datasets, respectively. This substantial improvement in efficiency compared to existing mainstream methods for extracting micro-expression subtle features effectively enhanced micro-expression recognition performance and increased the accuracy of high-precision micro-expression recognition. Consequently, this paper served as an important reference for researchers working on high-precision micro-expression recognition.

Dual-ATME: Dual-Branch Attention Network for Micro-Expression Recognition

Micro-expression recognition (MER) is challenging due to the difficulty of capturing the instantaneous and subtle motion changes of micro-expressions (MEs). Early works based on hand-crafted features extracted from prior knowledge showed some promising results, but have recently been replaced by deep learning methods based on the attention mechanism. However, with limited ME sample sizes, features extracted by these methods lack discriminative ME representations, in yet-to-be improved MER performance. This paper proposes the Dual-branch Attention Network (Dual-ATME) for MER to address the problem of ineffective single-scale features representing MEs. Specifically, Dual-ATME consists of two components: Hand-crafted Attention Region Selection (HARS) and Automated Attention Region Selection (AARS). HARS uses prior knowledge to manually extract features from regions of interest (ROIs). Meanwhile, AARS is based on attention mechanisms and extracts hidden information from data automatically. Finally, through similarity comparison and feature fusion, the dual-scale features could be used to learn ME representations effectively. Experiments on spontaneous ME datasets (including CASME II, SAMM, SMIC) and their composite dataset, MEGC2019-CD, showed that Dual-ATME achieves better, or more competitive, performance than the state-of-the-art MER methods.