Objective facial paralysis grading based onP face and eigenflow

Novel Baseline Facial Muscle Database Using Statistical Shape Modeling and In Silico Trials toward Decision Support for Facial Rehabilitation

Backgrounds and Objective: Facial palsy is a complex pathophysiological condition affecting the personal and professional lives of the involved patients. Sudden muscle weakness or paralysis needs to be rehabilitated to recover a symmetric and expressive face. Computer-aided decision support systems for facial rehabilitation have been developed. However, there is a lack of facial muscle baseline data to evaluate the patient states and guide as well as optimize the rehabilitation strategy. In this present study, we aimed to develop a novel baseline facial muscle database (static and dynamic behaviors) using the coupling between statistical shape modeling and in-silico trial approaches. Methods: 10,000 virtual subjects (5000 males and 5000 females) were generated from a statistical shape modeling (SSM) head model. Skull and muscle networks were defined so that they statistically fit with the head shapes. Two standard mimics: smiling and kissing were generated. The muscle strains of the lengths in neutral and mimic positions were computed and recorded thanks to the muscle insertion and attachment points on the animated head and skull meshes. For validation, five head and skull meshes were reconstructed from the five computed tomography (CT) image sets. Skull and muscle networks were then predicted from the reconstructed head meshes. The predicted skull meshes were compared with the reconstructed skull meshes based on the mesh-to-mesh distance metrics. The predicted muscle lengths were also compared with those manually defined on the reconstructed head and skull meshes. Moreover, the computed muscle lengths and strains were compared with those in our previous studies and the literature. Results: The skull prediction's median deviations from the CT-based models were 2.2236 mm, 2.1371 mm, and 2.1277 mm for the skull shape, skull mesh, and muscle attachment point regions, respectively. The median deviation of the muscle lengths was 4.8940 mm. The computed muscle strains were compatible with the reported values in our previous Kinect-based method and the literature. Conclusions: The development of our novel facial muscle database opens new avenues to accurately evaluate the facial muscle states of facial palsy patients. Based on the evaluated results, specific types of facial mimic rehabilitation exercises can also be selected optimally to train the target muscles. In perspective, the database of the computed muscle lengths and strains will be integrated into our available clinical decision support system for automatically detecting malfunctioning muscles and proposing patient-specific rehabilitation serious games.

Kinect-driven Patient-specific Head, Skull, and Muscle Network Modelling for Facial Palsy Patients

BACKGROUND AND OBJECTIVE

Facial palsy negatively affects both professional and personal life qualities of involved patients. Classical facial rehabilitation strategies can recover facial mimics into their normal and symmetrical movements and appearances. However, there is a lack of objective, quantitative, and in-vivo facial texture and muscle activation bio-feedbacks for personalizing rehabilitation programs and diagnosing recovering progresses. Consequently, this study proposed a novel patient-specific modelling method for generating a full patient specific head model from a visual sensor and then computing the facial texture and muscle activation in real-time for further clinical decision making.

METHODS

The modeling workflow includes (1) Kinect-to-head, (2) head-to-skull, and (3) muscle network definition & generation processes. In the Kinect-to-head process, subject-specific data acquired from a new user in neutral mimic were used for generating his/her geometrical head model with facial texture. In particular, a template head model was deformed to optimally fit with high-definition facial points acquired by the Kinect sensor. Moreover, the facial texture was also merged from his/her facial images in left, right, and center points of view. In the head-to-skull process, a generic skull model was deformed so that its shape was statistically fitted with his/her geometrical head model. In the muscle network definition & generation process, a muscle network was defined from the head and skull models for computing muscle strains during facial movements. Muscle insertion points and muscle attachment points were defined as vertex positions on the head model and the skull model respectively based on the standard facial anatomy. Three healthy subjects and two facial palsy patients were selected for validating the proposed method. In neutral positions, magnetic resonance imaging (MRI)-based head and skull models were compared with Kinect-based head and skull models. In mimic positions, infrared depth-based head models in smiling and [u]-pronouncing mimics were compared with appropriate animated Kinect-driven head models. The Hausdorff distance metric was used for these comparisons. Moreover, computed muscle lengths and strains in the tested facial mimics were validated with reported values in literature.

RESULTS

With the current hardware configuration, the patient-specific head model with skull and muscle network could be fast generated within 17.16±0.37s and animated in real-time with the framerate of 40 fps. In neutral positions, the best mean error was 1.91 mm for the head models and 3.21 mm for the skull models. On facial regions, the best mean errors were 1.53 mm and 2.82 mm for head and skull models respectively. On muscle insertion/attachment point regions, the best mean errors were 1.09 mm and 2.16 mm for head and skull models respectively. In mimic positions, these errors were 2.02 mm in smiling mimics and 2.00 mm in [u]-pronouncing mimics for the head models on facial regions. All above error values were computed on a one-time validation procedure. Facial muscles exhibited muscle shortening and muscle elongating for smiling and pronunciation of sound [u] respectively. Extracted muscle features (i.e. muscle length and strain) are in agreement with experimental and literature data.

CONCLUSIONS

This study proposed a novel modeling method for fast generating and animating patient-specific biomechanical head model with facial texture and muscle activation bio-feedbacks. The Kinect-driven muscle strains could be applied for further real-time muscle-oriented facial paralysis grading and other facial analysis applications.

A Simple, Objective, and Mathematical Grading Scale for the Assessment of Facial Nerve Palsy

OBJECTIVES

It is imperative to acquire a simple, objective, and mathematical method for the assessment of facial nerve palsy which can be universally accepted and implemented. A grading scale which is convenient, continuous and economical was attempted for the first time for global and region-specific assessment of facial nerve palsy.

STUDY DESIGN

Hospital-based observational study.

SETTING

Medical college hospital.

PATIENTS

Ten normal subjects and 51 patients with facial paralysis.

INTERVENTIONS

Patients with facial nerve palsy were graded according to the revised version of House-Brackmann grading system (HBGS-2) and a newly proposed grading system.

MAIN OUTCOME MEASURES

The results of the present study were compared with the HBGS-2. Data were analyzed using SPSS-17 (IBM Corporation, New York) for descriptive statistics, normality test, Wilcoxon signed-rank test, and Mann-Whitney U test.

RESULTS

The mean time spent on recording measurements was 288 seconds. For the new method and HBGS-2, the modes were graded 3 and 4, corresponding to incomplete facial paralysis. The Kolmogorov-Smirnov normality and Wilcoxon signed rank tests were found significant. In Mann-Whitney U test, probability value indicated that grades of new scale were similar to grades of HBGS-2.

CONCLUSION

The proposed simple, objective and mathematical (SOM) method of grading facial nerve palsy is convenient and provides global and regional continuous percentage that can monitor the progress and classify the patients with facial paralysis into six-point grades based on severity. This system was having substantial compatibility with HBGS-2 grading. For further validity, multi-center study with a larger sample of patients would be required.

paraFaceTest: an ensemble of regression tree-based facial features extraction for efficient facial paralysis classification

Background

Facial paralysis (FP) is a neuromotor dysfunction that losses voluntary muscles movement in one side of the human face. As the face is the basic means of social interactions and emotional expressions among humans, individuals afflicted can often be introverted and may develop psychological distress, which can be even more severe than the physical disability. This paper addresses the problem of objective facial paralysis evaluation.

Methods

We present a novel approach for objective facial paralysis evaluation and classification, which is crucial for deciding the medical treatment scheme. For FP classification, in particular, we proposed a method based on the ensemble of regression trees to efficiently extract facial salient points and detect iris or sclera boundaries. We also employ 2^nd degree polynomial of parabolic function to improve Daugman’s algorithm for detecting occluded iris boundaries, thereby allowing us to efficiently get the area of the iris. The symmetry score of each face is measured by calculating the ratio of both iris area and the distances between the key points in both sides of the face. We build a model by employing hybrid classifier that discriminates healthy from unhealthy subjects and performs FP classification.

Results

Objective analysis was conducted to evaluate the performance of the proposed method. As we explore the effect of data augmentation using publicly available datasets of facial expressions, experiments reveal that the proposed approach demonstrates efficiency.

Conclusions

Extraction of iris and facial salient points on images based on ensemble of regression trees along with our hybrid classifier (classification tree plus regularized logistic regression) provides a more improved way of addressing FP classification problem. It addresses the common limiting factor introduced in the previous works, i.e. having the greater sensitivity to subjects exposed to peculiar facial images, whereby improper identification of initial evolving curve for facial feature segmentation results to inaccurate facial feature extraction. Leveraging ensemble of regression trees provides accurate salient points extraction, which is crucial for revealing the significant difference between the healthy and the palsy side when performing different facial expressions.

Efficient quantitative assessment of facial paralysis using iris segmentation and active contour-based key points detection with hybrid classifier

BACKGROUND

Facial palsy or paralysis (FP) is a symptom that loses voluntary muscles movement in one side of the human face, which could be very devastating in the part of the patients. Traditional methods are solely dependent to clinician's judgment and therefore time consuming and subjective in nature. Hence, a quantitative assessment system becomes apparently invaluable for physicians to begin the rehabilitation process; and to produce a reliable and robust method is challenging and still underway.

METHODS

We introduce a novel approach for a quantitative assessment of facial paralysis that tackles classification problem for FP type and degree of severity. Specifically, a novel method of quantitative assessment is presented: an algorithm that extracts the human iris and detects facial landmarks; and a hybrid approach combining the rule-based and machine learning algorithm to analyze and prognosticate facial paralysis using the captured images. A method combining the optimized Daugman's algorithm and Localized Active Contour (LAC) model is proposed to efficiently extract the iris and facial landmark or key points. To improve the performance of LAC, appropriate parameters of initial evolving curve for facial features' segmentation are automatically selected. The symmetry score is measured by the ratio between features extracted from the two sides of the face. Hybrid classifiers (i.e. rule-based with regularized logistic regression) were employed for discriminating healthy and unhealthy subjects, FP type classification, and for facial paralysis grading based on House-Brackmann (H-B) scale.

RESULTS

Quantitative analysis was performed to evaluate the performance of the proposed approach. Experiments show that the proposed method demonstrates its efficiency.

CONCLUSIONS

Facial movement feature extraction on facial images based on iris segmentation and LAC-based key point detection along with a hybrid classifier provides a more efficient way of addressing classification problem on facial palsy type and degree of severity. Combining iris segmentation and key point-based method has several merits that are essential for our real application. Aside from the facial key points, iris segmentation provides significant contribution as it describes the changes of the iris exposure while performing some facial expressions. It reveals the significant difference between the healthy side and the severe palsy side when raising eyebrows with both eyes directed upward, and can model the typical changes in the iris region.

Automated Facial Image Analysis

The purpose of this study was to evaluate the ability of Automated Facial Image Analysis (AFA) to detect changes in facial motion after Botox injections in patients with facial nerve disorders accompanied by abnormal muscle activity. Eight subjects received Botox for oral to ocular synkinesis (n = 6), ocular to oral synkinesis (n = 1), and/or depressor anguli oris overactivity (n = 3). Subjects were video-recorded during 2 directed facial action tasks before and after Botox treatment. AFA measurement and Facial Grading System (FGS) scores were used to evaluate the effects of Botox. After Botox, AFA detected a decrease in abnormal movements of the eyelids in all patients with oral to ocular synkinesis, a decrease in oral commissure movement for the patients with ocular to oral synkinesis, and an increase in oral commissure movement in all patients with depressor overactivity. The FGS scores failed to demonstrate any change in facial movement for the case of ocular to oral synkinesis and for 2 cases of depressor overactivity. AFA enables recognition of subtle changes in facial movement that may not be adequately measured by observer based ratings of facial function.