Few-Shot Learning with a Novel Voronoi Tessellation-Based Image Augmentation Method for Facial Palsy Detection

Facial expressions to identify post-stroke: A pilot study

BACKGROUND AND OBJECTIVE

Timely stroke treatment can limit brain damage and improve outcomes, which depends on early recognition of the symptoms. However, stroke cases are often missed by the first respondent paramedics. One of the earliest external symptoms of stroke is based on facial expressions.

METHODS

We propose a computerized analysis of facial expressions using action units to distinguish between Post-Stroke and healthy people. Action units enable analysis of subtle and specific facial movements and are interpretable to the facial expressions. The RGB videos from the Toronto Neuroface Dataset, which were recorded during standard orofacial examinations of 14 people with post-stroke (PS) and 11 healthy controls (HC) were used in this study. Action units were computed using XGBoost which was trained using HC, and classified using regression analysis for each of the nine facial expressions. The analysis was performed without manual intervention.

RESULTS

The results were evaluated using leave-one-our validation. The accuracy was 82% for Kiss and Spread, with the best sensitivity of 91% in the differentiation of PS and HC. The features corresponding to mouth muscles were most suitable.

CONCLUSIONS

This pilot study has shown that our method can detect PS based on two simple facial expressions. However, this needs to be tested in real-world conditions, with people of different ethnicities and smartphone use. The method has the potential for a computerized assessment of the videos for use by the first respondents using a smartphone to perform screening tests, which can facilitate the timely start of the treatment.

Analyzing Facial Asymmetry in Alzheimer's Dementia Using Image-Based Technology

Several studies have demonstrated accelerated brain aging in Alzheimer's dementia (AD). Previous studies have also reported that facial asymmetry increases with age. Because obtaining facial images is much easier than obtaining brain images, the aim of this work was to investigate whether AD exhibits accelerated aging patterns in facial asymmetry. We developed new facial asymmetry measures to compare Alzheimer's patients with healthy controls. A three-dimensional camera was used to capture facial images, and 68 facial landmarks were identified using an open-source machine-learning algorithm called OpenFace. A standard image registration method was used to align the three-dimensional original and mirrored facial images. This study used the registration error, representing landmark superimposition asymmetry distances, to examine 29 pairs of landmarks to characterize facial asymmetry. After comparing the facial images of 150 patients with AD with those of 150 age- and sex-matched non-demented controls, we found that the asymmetry of 20 landmarks was significantly different in AD than in the controls (p < 0.05). The AD-linked asymmetry was concentrated in the face edge, eyebrows, eyes, nostrils, and mouth. Facial asymmetry evaluation may thus serve as a tool for the detection of AD.

Snow Leopard Appearance of Subcutaneous Panniculitis such as T-cell Lymphoma on 18F-FDG PET/CT

Subcutaneous panniculitis such as T-cell lymphoma (SPTCL) is a very rare disorder. Patients usually present with multiple subcutaneous nodules on the extremities without visceral disease. Dissemination to extra-cutaneous sites is unusual. Only a few cases of SPTCL have been reported in the literature describing the findings of ¹⁸F-fluorodeoxyglucose (FDG) positron emission tomography (PET). Here, we represent an interesting and unusual case of diffuse SPTCL with snow Leopard skin appearance on ¹⁸F-FDG PET/computed tomography.

The Feasibility of An Automatical Facial Evaluation System Providing Objective and Reliable Results for Facial Palsy

Facial palsy would lead to a series of physical and mental problems, as facial function plays an important role in various aspects of daily life. However, the current strategies for evaluating facial function relied heavily on raters and the results varied from the experience of raters. Thus, an objective and accurate facial evaluation system is always claimed. In this study, a customized automatical facial evaluation system (AFES) was proposed, which might have the potential to be employed as an adjunctive and efficient assessing method in clinic. In order to investigate the feasibility of AFES, ninety-two participants with facial palsy were recruited and received scale-based subjective manual evaluation (including mHBGS and mSFGS) and objective automatical evaluation of AFES (including aHBGS, aSFGS and indicators of facial regional features) at enrollment and after two weeks. The correlations between the results of the two methods were analyzed and the participants were stratified according to the severity of facial function for further analyses. Strong positive correlations between manual and automatical HBGS and SFGS were observed and higher correlations were reported in the participants with normal-mild and moderate facial palsy. Significant improvements in clinical scales and indicator of eye synkinesis were found in forty-two participants in two weeks. Furthermore, some of the indicators were correlated with scale scores (I4, I7) and one of them presented a significant change between the baseline evaluation and follow-up evaluation (I7). According to the results, AFES could be considered as a viable method to perform objective and reliable evaluation for patients with facial palsy and provide clarified results for prognosis.

Automatic Facial Palsy Diagnosis as a Classification Problem Using Regional Information Extracted from a Photograph

The incapability to move the facial muscles is known as facial palsy, and it affects various abilities of the patient, for example, performing facial expressions. Recently, automatic approaches aiming to diagnose facial palsy using images and machine learning algorithms have emerged, focusing on providing an objective evaluation of the paralysis severity. This research proposes an approach to analyze and assess the lesion severity as a classification problem with three levels: healthy, slight, and strong palsy. The method explores the use of regional information, meaning that only certain areas of the face are of interest. Experiments carrying on multi-class classification tasks are performed using four different classifiers to validate a set of proposed hand-crafted features. After a set of experiments using this methodology on available image databases, great results are revealed (up to 95.61% of correct detection of palsy patients and 95.58% of correct assessment of the severity level). This perspective leads us to believe that the analysis of facial paralysis is possible with partial occlusions if face detection is accomplished and facial features are obtained adequately. The results also show that our methodology is suited to operate with other databases while attaining high performance, even though the image conditions are different and the participants do not perform equivalent facial expressions.

Word Embedding Distribution Propagation Graph Network for Few-Shot Learning

Few-shot learning (FSL) is of great significance to the field of machine learning. The ability to learn and generalize using a small number of samples is an obvious distinction between artificial intelligence and humans. In the FSL domain, most graph neural networks (GNNs) focus on transferring labeled sample information to an unlabeled query sample, ignoring the important role of semantic information during the classification process. Our proposed method embeds semantic information of classes into a GNN, creating a word embedding distribution propagation graph network (WPGN) for FSL. We merge the attention mechanism with our backbone network, use the Mahalanobis distance to calculate the similarity of classes, select the Funnel ReLU (FReLU) function as the activation function of the Transform layer, and update the point graph and word embedding distribution graph. In extensive experiments on FSL benchmarks, compared with the baseline model, the accuracy of the WPGN on the 5-way-1/2/5 shot tasks increased by 9.03, 4.56, and 4.15%, respectively.

Data Augmentation for Audio-Visual Emotion Recognition with an Efficient Multimodal Conditional GAN

Audio-visual emotion recognition is the research of identifying human emotional states by combining the audio modality and the visual modality simultaneously, which plays an important role in intelligent human-machine interactions. With the help of deep learning, previous works have made great progress for audio-visual emotion recognition. However, these deep learning methods often require a large amount of data for training. In reality, data acquisition is difficult and expensive, especially for the multimodal data with different modalities. As a result, the training data may be in the low-data regime, which cannot be effectively used for deep learning. In addition, class imbalance may occur in the emotional data, which can further degrade the performance of audio-visual emotion recognition. To address these problems, we propose an efficient data augmentation framework by designing a multimodal conditional generative adversarial network (GAN) for audio-visual emotion recognition. Specifically, we design generators and discriminators for audio and visual modalities. The category information is used as their shared input to make sure our GAN can generate fake data of different categories. In addition, the high dependence between the audio modality and the visual modality in the generated multimodal data is modeled based on Hirschfeld-Gebelein-Rényi (HGR) maximal correlation. In this way, we relate different modalities in the generated data to approximate the real data. Then, the generated data are used to augment our data manifold. We further apply our approach to deal with the problem of class imbalance. To the best of our knowledge, this is the first work to propose a data augmentation strategy with a multimodal conditional GAN for audio-visual emotion recognition. We conduct a series of experiments on three public multimodal datasets, including eNTERFACE’05, RAVDESS, and CMEW. The results indicate that our multimodal conditional GAN has high effectiveness for data augmentation of audio-visual emotion recognition.

A Decision Support System for Face Sketch Synthesis Using Deep Learning and Artificial Intelligence

The recent development in the area of IoT technologies is likely to be implemented extensively in the next decade. There is a great increase in the crime rate, and the handling officers are responsible for dealing with a broad range of cyber and Internet issues during investigation. IoT technologies are helpful in the identification of suspects, and few technologies are available that use IoT and deep learning together for face sketch synthesis. Convolutional neural networks (CNNs) and other constructs of deep learning have become major tools in recent approaches. A new-found architecture of the neural network is anticipated in this work. It is called Spiral-Net, which is a modified version of U-Net fto perform face sketch synthesis (the phase is known as the compiler network C here). Spiral-Net performs in combination with a pre-trained Vgg-19 network called the feature extractor F. It first identifies the top n matches from viewed sketches to a given photo. F is again used to formulate a feature map based on the cosine distance of a candidate sketch formed by C from the top n matches. A customized CNN configuration (called the discriminator D) then computes loss functions based on differences between the candidate sketch and the feature. Values of these loss functions alternately update C and F. The ensemble of these nets is trained and tested on selected datasets, including CUFS, CUFSF, and a part of the IIT photo–sketch dataset. Results of this modified U-Net are acquired by the legacy NLDA (1998) scheme of face recognition and its newer version, OpenBR (2013), which demonstrate an improvement of 5% compared with the current state of the art in its relevant domain.

Pose Estimation of Driver’s Head Panning Based on Interpolation and Motion Vectors under a Boosting Framework

Over the last decade, a driver’s distraction has gained popularity due to its increased significance and high impact on road accidents. Various factors, such as mood disorder, anxiety, nervousness, illness, loud music, and driver’s head rotation, contribute significantly to causing a distraction. Many solutions have been proposed to address this problem; however, various aspects of it are still unresolved. The study proposes novel geometric and spatial scale-invariant features under a boosting framework for detecting a driver’s distraction due to the driver’s head panning. These features are calculated using facial landmark detection algorithms, including the Active Shape Model (ASM) and Boosted Regression with Markov Networks (BoRMaN). The proposed approach is compared with six existing state-of-the-art approaches using four benchmark datasets, including DrivFace dataset, Boston University (BU) dataset, FT-UMT dataset, and Pointing’04 dataset. The proposed approach outperforms the existing approaches achieving an accuracy of 94.43%, 92.08%, 96.63%, and 83.25% on standard datasets.

Human, All Too Human? An All-Around Appraisal of the "Artificial Intelligence Revolution" in Medical Imaging

Artificial intelligence (AI) has seen dramatic growth over the past decade, evolving from a niche super specialty computer application into a powerful tool which has revolutionized many areas of our professional and daily lives, and the potential of which seems to be still largely untapped. The field of medicine and medical imaging, as one of its various specialties, has gained considerable benefit from AI, including improved diagnostic accuracy and the possibility of predicting individual patient outcomes and options of more personalized treatment. It should be noted that this process can actively support the ongoing development of advanced, highly specific treatment strategies (e.g., target therapies for cancer patients) while enabling faster workflow and more efficient use of healthcare resources. The potential advantages of AI over conventional methods have made it attractive for physicians and other healthcare stakeholders, raising much interest in both the research and the industry communities. However, the fast development of AI has unveiled its potential for disrupting the work of healthcare professionals, spawning concerns among radiologists that, in the future, AI may outperform them, thus damaging their reputations or putting their jobs at risk. Furthermore, this development has raised relevant psychological, ethical, and medico-legal issues which need to be addressed for AI to be considered fully capable of patient management. The aim of this review is to provide a brief, hopefully exhaustive, overview of the state of the art of AI systems regarding medical imaging, with a special focus on how AI and the entire healthcare environment should be prepared to accomplish the goal of a more advanced human-centered world.