MESNet: A Convolutional Neural Network for Spotting Multi-Scale Micro-Expression Intervals in Long Videos

SFTNet: A microexpression-based method for depression detection

BACKGROUND AND OBJECTIVES

Depression is a typical mental illness, and early screening can effectively prevent exacerbation of the condition. Many studies have found that the expressions of depressed patients are different from those of other subjects, and microexpressions have been used in the clinical detection of mental illness. However, there are few methods for the automatic detection of depression based on microexpressions.

METHODS

A new dataset of 156 participants (76 in the case group and 80 in the control group) was created. All data were collected in the context of a new emotional stimulation experiment and doctor-patient conversation. We first analyzed the Average Number of Occurrences (ANO) and Average Duration (AD) of facial expressions in the case group and the control group. Then, we proposed a two-stream model SFTNet for identifying depression based on microexpressions, which consists of a single-temporal network (STNet) and a full-temporal network (FTNet). STNet is used to extract features from facial images at a single time node, FTNet is used to extract features from all-time nodes, and the decision network combines the two features to identify depression through decision fusion. The code for SFTNet is available at https://github.com/muzixingyun/SFTNet.

RESULTS

We found that the AD of all subjects was less than 20 frames (2/3 seconds) and that the facial expressions of the control group were richer. SFTNet achieved excellent results on the emotional stimulus experimental dataset, with Accuracy, Precision and Recall of 0.873, 0.888 and 0.846, respectively. We also conducted experiments on the doctor-patient conversation dataset, and the Accuracy, Precision and Recall were 0.829, 0.817 and 0.837, respectively. SFTNet can also be applied to microexpression detection task with more accuracy than SOTA models.

CONCLUSIONS

In the emotional stimulation experiment, the subjects in the case group are more likely to show negative emotions. Compared to SOTA models, our depression detection method is more accurate and can assist doctors in the diagnosis of depression.

Inverse Design of Nanophotonic Devices Using Generative Adversarial Networks with the Sim-NN Model and Self-Attention Mechanism

The inverse design method based on a generative adversarial network (GAN) combined with a simulation neural network (sim-NN) and the self-attention mechanism is proposed in order to improve the efficiency of GAN for designing nanophotonic devices. The sim-NN can guide the model to produce more accurate device designs via the spectrum comparison, whereas the self-attention mechanism can help to extract detailed features of the spectrum by exploring their global interconnections. The nanopatterned power splitter with a 2 μm × 2 μm interference region is designed as an example to obtain the average high transmission (>94%) and low back-reflection (<0.5%) over the broad wavelength range of 1200~1650 nm. As compared to other models, this method can produce larger proportions of high figure-of-merit devices with various desired power-splitting ratios.

CAS(ME)3: A Third Generation Facial Spontaneous Micro-Expression Database With Depth Information and High Ecological Validity

Micro-expression (ME) is a significant non-verbal communication clue that reveals one person's genuine emotional state. The development of micro-expression analysis (MEA) has just gained attention in the last decade. However, the small sample size problem constrains the use of deep learning on MEA. Besides, ME samples distribute in six different databases, leading to database bias. Moreover, the ME database development is complicated. In this article, we introduce a large-scale spontaneous ME database: CAS(ME) 3. The contribution of this article is summarized as follows: (1) CAS(ME) 3 offers around 80 hours of videos with over 8,000,000 frames, including manually labeled 1,109 MEs and 3,490 macro-expressions. Such a large sample size allows effective MEA method validation while avoiding database bias. (2) Inspired by psychological experiments, CAS(ME) 3 provides the depth information as an additional modality unprecedentedly, contributing to multi-modal MEA. (3) For the first time, CAS(ME) 3 elicits ME with high ecological validity using the mock crime paradigm, along with physiological and voice signals, contributing to practical MEA. (4) Besides, CAS(ME) 3 provides 1,508 unlabeled videos with more than 4,000,000 frames, i.e., a data platform for unsupervised MEA methods. (5) Finally, we demonstrate the effectiveness of depth information by the proposed depth flow algorithm and RGB-D information.

Differences in brain activations between micro- and macro-expressions based on electroencephalography

Micro-expressions can reflect an individual's subjective emotions and true mental state and are widely used in the fields of mental health, justice, law enforcement, intelligence, and security. However, the current approach based on image and expert assessment-based micro-expression recognition technology has limitations such as limited application scenarios and time consumption. Therefore, to overcome these limitations, this study is the first to explore the brain mechanisms of micro-expressions and their differences from macro-expressions from a neuroscientific perspective. This can be a foundation for micro-expression recognition based on EEG signals. We designed a real-time supervision and emotional expression suppression (SEES) experimental paradigm to synchronously collect facial expressions and electroencephalograms. Electroencephalogram signals were analyzed at the scalp and source levels to determine the temporal and spatial neural patterns of micro- and macro-expressions. We found that micro-expressions were more strongly activated in the premotor cortex, supplementary motor cortex, and middle frontal gyrus in frontal regions under positive emotions than macro-expressions. Under negative emotions, micro-expressions were more weakly activated in the somatosensory cortex and corneal gyrus regions than macro-expressions. The activation of the right temporoparietal junction (rTPJ) was stronger in micro-expressions under positive than negative emotions. The reason for this difference is that the pathways of facial control are different; the production of micro-expressions under positive emotion is dependent on the control of the face, while micro-expressions under negative emotions are more dependent on the intensity of the emotion.

Machine learning for early discrimination between transient and persistent acute kidney injury in critically ill patients with sepsis

Acute kidney injury (AKI) is commonly present in critically ill patients with sepsis. Early prediction of short-term reversibility of AKI is beneficial to risk stratification and clinical treatment decision. The study sought to use machine learning methods to discriminate between transient and persistent sepsis-associated AKI. Septic patients who developed AKI within the first 48 h after ICU admission were identified from the Medical Information Mart for Intensive Care III database. AKI was classified as transient or persistent according to the Acute Disease Quality Initiative workgroup consensus. Five prediction models using logistic regression, random forest, support vector machine, artificial neural network and extreme gradient boosting were constructed, and their performance was evaluated by out-of-sample testing. A simplified risk prediction model was also derived based on logistic regression and features selected by machine learning algorithms. A total of 5984 septic patients with AKI were included, 3805 (63.6%) of whom developed persistent AKI. The artificial neural network and logistic regression models achieved the highest area under the receiver operating characteristic curve (AUC) among the five machine learning models (0.76, 95% confidence interval [CI] 0.74-0.78). The simplified 14-variable model showed adequate discrimination, with the AUC being 0.76 (95% CI 0.73-0.78). At the optimal cutoff of 0.63, the sensitivity and specificity of the simplified model were 63% and 76% respectively. In conclusion, a machine learning-based simplified prediction model including routine clinical variables could be used to differentiate between transient and persistent AKI in critically ill septic patients. An easy-to-use risk calculator can promote its widespread application in daily clinical practice.

Evolutionary warning system for COVID-19 severity: Colony predation algorithm enhanced extreme learning machine

Coronavirus Disease 2019 (COVID-19) was distributed globally at the end of December 2019 due to severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). Early diagnosis and successful COVID-19 assessment are missing, clinical care is ineffective, and deaths are high. In this study, we investigate whether the level of biochemical indicators helps to discriminate and classify the severity of the COVID-19 using the machine learning method. This research creates an efficient intelligence method for the diagnosis of COVID-19 from the perspective of biochemical indexes. The framework is proposed by integrating an enhanced new stochastic called the colony predation algorithm (CPA) with a kernel extreme learning machine (KELM), abbreviated as ECPA-KELM. The core feature of the approach is the ECPA algorithm which incorporates the two main operators that have been abstained from the grey wolf optimizer and moth-flame optimizer to improve and restore the CPA research functions and are simultaneously used to optimize the parameters and to select features for KELM. The ECPA output is checked thoroughly using IEEE CEC2017 benchmark to verify the capacity of the proposed methodology. Finally, in the diagnosis of COVID-19 using biochemical indexes, the designed ECPA-KELM model and other competing KELM models based on other optimization are used. Checking statistical results will display improved predictive properties for all metrics and higher stability. ECPA-KELM can also be used to discriminate and classify the severity of the COVID-19 as a possible computer-aided method and provide effective early warning for the treatment and diagnosis of COVID-19.

Evolving fuzzy k-nearest neighbors using an enhanced sine cosine algorithm: Case study of lupus nephritis

Because of its simplicity and effectiveness, fuzzy K-nearest neighbors (FKNN) is widely used in literature. The parameters have an essential impact on the performance of FKNN. Hence, the parameters need to be attuned to suit different problems. Also, choosing more representative features can enhance the performance of FKNN. This research proposes an improved optimization technique based on the sine cosine algorithm (LSCA), which introduces a linear population size reduction mechanism for enhancing the original algorithm's performance. Moreover, we developed an FKNN model based on the LSCA, it simultaneously performs feature selection and parameter optimization. Firstly, the search performance of LSCA is verified on the IEEE CEC2017 benchmark test function compared to the classical and improved algorithms. Secondly, the validity of the LSCA-FKNN model is verified on three medical datasets. Finally, we used the proposed LSCA-FKNN to predict lupus nephritis classes, and the model showed competitive results. The paper will be supported by an online web service for any question at https://aliasgharheidari.com.