Novel Baseline Facial Muscle Database Using Statistical Shape Modeling and In Silico Trials toward Decision Support for Facial Rehabilitation

Backgrounds and Objective: Facial palsy is a complex pathophysiological condition affecting the personal and professional lives of the involved patients. Sudden muscle weakness or paralysis needs to be rehabilitated to recover a symmetric and expressive face. Computer-aided decision support systems for facial rehabilitation have been developed. However, there is a lack of facial muscle baseline data to evaluate the patient states and guide as well as optimize the rehabilitation strategy. In this present study, we aimed to develop a novel baseline facial muscle database (static and dynamic behaviors) using the coupling between statistical shape modeling and in-silico trial approaches. Methods: 10,000 virtual subjects (5000 males and 5000 females) were generated from a statistical shape modeling (SSM) head model. Skull and muscle networks were defined so that they statistically fit with the head shapes. Two standard mimics: smiling and kissing were generated. The muscle strains of the lengths in neutral and mimic positions were computed and recorded thanks to the muscle insertion and attachment points on the animated head and skull meshes. For validation, five head and skull meshes were reconstructed from the five computed tomography (CT) image sets. Skull and muscle networks were then predicted from the reconstructed head meshes. The predicted skull meshes were compared with the reconstructed skull meshes based on the mesh-to-mesh distance metrics. The predicted muscle lengths were also compared with those manually defined on the reconstructed head and skull meshes. Moreover, the computed muscle lengths and strains were compared with those in our previous studies and the literature. Results: The skull prediction's median deviations from the CT-based models were 2.2236 mm, 2.1371 mm, and 2.1277 mm for the skull shape, skull mesh, and muscle attachment point regions, respectively. The median deviation of the muscle lengths was 4.8940 mm. The computed muscle strains were compatible with the reported values in our previous Kinect-based method and the literature. Conclusions: The development of our novel facial muscle database opens new avenues to accurately evaluate the facial muscle states of facial palsy patients. Based on the evaluated results, specific types of facial mimic rehabilitation exercises can also be selected optimally to train the target muscles. In perspective, the database of the computed muscle lengths and strains will be integrated into our available clinical decision support system for automatically detecting malfunctioning muscles and proposing patient-specific rehabilitation serious games.

The Feasibility of An Automatical Facial Evaluation System Providing Objective and Reliable Results for Facial Palsy

Facial palsy would lead to a series of physical and mental problems, as facial function plays an important role in various aspects of daily life. However, the current strategies for evaluating facial function relied heavily on raters and the results varied from the experience of raters. Thus, an objective and accurate facial evaluation system is always claimed. In this study, a customized automatical facial evaluation system (AFES) was proposed, which might have the potential to be employed as an adjunctive and efficient assessing method in clinic. In order to investigate the feasibility of AFES, ninety-two participants with facial palsy were recruited and received scale-based subjective manual evaluation (including mHBGS and mSFGS) and objective automatical evaluation of AFES (including aHBGS, aSFGS and indicators of facial regional features) at enrollment and after two weeks. The correlations between the results of the two methods were analyzed and the participants were stratified according to the severity of facial function for further analyses. Strong positive correlations between manual and automatical HBGS and SFGS were observed and higher correlations were reported in the participants with normal-mild and moderate facial palsy. Significant improvements in clinical scales and indicator of eye synkinesis were found in forty-two participants in two weeks. Furthermore, some of the indicators were correlated with scale scores (I4, I7) and one of them presented a significant change between the baseline evaluation and follow-up evaluation (I7). According to the results, AFES could be considered as a viable method to perform objective and reliable evaluation for patients with facial palsy and provide clarified results for prognosis.

Emerging artificial intelligence applications in otological imaging

PURPOSE OF REVIEW

To highlight the recent literature on artificial intelligence (AI) pertaining to otological imaging and to discuss future directions, obstacles and opportunities.

RECENT FINDINGS

The main themes in the recent literature centre around automated otoscopic image diagnosis and automated image segmentation for application in virtual reality surgical simulation and planning. Other applications that have been studied include identification of tinnitus MRI biomarkers, facial palsy analysis, intraoperative augmented reality systems, vertigo diagnosis and endolymphatic hydrops ratio calculation in Meniere's disease. Studies are presently at a preclinical, proof-of-concept stage.

SUMMARY

The recent literature on AI in otological imaging is promising and demonstrates the future potential of this technology in automating certain imaging tasks in a healthcare environment of ever-increasing demand and workload. Some studies have shown equivalence or superiority of the algorithm over physicians, albeit in narrowly defined realms. Future challenges in developing this technology include the compilation of large high quality annotated datasets, fostering strong collaborations between the health and technology sectors, testing the technology within real-world clinical pathways and bolstering trust among patients and physicians in this new method of delivering healthcare.

Few-Shot Learning with a Novel Voronoi Tessellation-Based Image Augmentation Method for Facial Palsy Detection

Face palsy has adverse effects on the appearance of a person and has negative social and functional consequences on the patient. Deep learning methods can improve face palsy detection rate, but their efficiency is limited by insufficient data, class imbalance, and high misclassification rate. To alleviate the lack of data and improve the performance of deep learning models for palsy face detection, data augmentation methods can be used. In this paper, we propose a novel Voronoi decomposition-based random region erasing (VDRRE) image augmentation method consisting of partitioning images into randomly defined Voronoi cells as an alternative to rectangular based random erasing method. The proposed method augments the image dataset with new images, which are used to train the deep neural network. We achieved an accuracy of 99.34% using two-shot learning with VDRRE augmentation on palsy faces from Youtube Face Palsy (YFP) dataset, while normal faces are taken from Caltech Face Database. Our model shows an improvement over state-of-the-art methods in the detection of facial palsy from a small dataset of face images.

Facial Paralysis Detection on Images Using Key Point Analysis

The inability to move the muscles of the face on one or both sides is known as facial paralysis, which may affect the ability of the patient to speak, blink, swallow saliva, eat, or communicate through natural facial expressions. The well-being of the patient could also be negatively affected. Computer-based systems as a means to detect facial paralysis are important in the development of standardized tools for medical assessment, treatment, and monitoring; additionally, they are expected to provide user-friendly tools for patient monitoring at home. In this work, a methodology to detect facial paralysis in a face photograph is proposed. A system consisting of three modules—facial landmark extraction, facial measure computation, and facial paralysis classification—was designed. Our facial measures aim to identify asymmetry levels within the face elements using facial landmarks, and a binary classifier based on a multi-layer perceptron approach provides an output label. The Weka suite was selected to design the classifier and implement the learning algorithm. Tests on publicly available databases reveal outstanding classification results on images, showing that our methodology that was used to design a binary classifier can be expanded to other databases with great results, even if the participants do not execute similar facial expressions.

Detection of facial landmarks by a convolutional neural network in patients with oral and maxillofacial disease

Facial nerve dysfunction is common in patients with Bell's palsy, trauma, tumour, or iatrogenic injuries. Imaging assessment is the most convenient method for patients and their treating physician. With developments in artificial intelligence (AI), manual work will be replaced. In this study, a database of facial images of patients with oral and maxillofacial diseases was set up to develop a facial nerve functional assessment system based on AI. This database was then used to evaluate the accuracy of a state-of-the-art algorithm for facial landmark detection named 'HRNet'. Utilizing this database and with appropriate human intervention, HRNet was used in facial annotation. The accuracy of annotations was evaluated through the normalized mean error. A total of 912 images were collected from 300 people; 546 of these images had abnormal features including defects, swelling, scars, or facial paralysis. The accuracy for the abnormal group was lower than that for the normal group before and after training, but improvements in accuracy were identified in both groups post-training. In conclusion, this new database demonstrates the ability of HRNet to localize facial landmarks in patients with oral and maxillofacial diseases. More images for training should be added to this database to diversify it in the future.

Comparison of Facial Muscle Activation Patterns Between Healthy and Bell's Palsy Subjects Using High-Density Surface Electromyography

Facial muscle activities are essential for the appearance and communication of human beings. Therefore, exploring the activation patterns of facial muscles can help understand facial neuromuscular disorders such as Bell’s palsy. Given the irregular shape of the facial muscles as well as their different locations, it should be difficult to detect the activities of whole facial muscles with a few electrodes. In this study, a high-density surface electromyogram (HD sEMG) system with 90 electrodes was used to record EMG signals of facial muscles in both healthy and Bell’s palsy subjects when they did different facial movements. The electrodes were arranged in rectangular arrays covering the forehead and cheek regions of the face. The muscle activation patterns were shown on maps, which were constructed from the Root Mean Square (RMS) values of all the 90-channel EMG recordings. The experimental results showed that the activation patterns of facial muscles were distinct during doing different facial movements and the activated muscle regions could be clearly observed. Moreover, two features of the activation patterns, 2D correlation coefficient (corr2) and Centre of Gravity (CG) were extracted to quantify the spatial symmetry and the location of activated muscle regions respectively. Furthermore, the deviation of activated muscle regions on the paralyzed side of a face compared to the healthy side was quantified by calculating the distance between two sides of CGs. The results revealed that corr2 of the activated facial muscle region (classified into forehead region and cheek region) in Bell’s palsy subjects was significantly (p < 0.05) lower than that in healthy subjects, while CG distance of activated facial region in Bell’s palsy subjects was significantly (p < 0.05) higher than that in healthy subjects. The correlation between corr2 of these regions and Bell’s palsy [assessed by the Facial Nerve Grading Scale (FNGS) 2.0] was also significant (p < 0.05) in Bell’s palsy subjects. The spatial information on activated muscle regions may be useful in the diagnosis and treatment of Bell’s palsy in the future.

Region Based Parallel Hierarchy Convolutional Neural Network for Automatic Facial Nerve Paralysis Evaluation

In this article, we propose a parallel hierarchy convolutional neural network (PHCNN) combining a Long Short-Term Memory (LSTM) network structure to quantitatively assess the grading of facial nerve paralysis (FNP) by considering the region-based asymmetric facial features and temporal variation of the image sequences. FNP, such as Bell's palsy, is the most common facial symptom of neuromotor dysfunctions. It causes the weakness of facial muscles for the normal emotional expression and movements. The subjective judgement by clinicians completely depends on individual experience, which may not lead to a uniform evaluation. Existing computer-aided methods mainly rely on some complicated imaging equipment, which is complicated and expensive for facial functional rehabilitation. Compared with the subjective judgment and complex imaging processing, the objective and intelligent measurement can potentially avoid this issue. Considering dynamic variation in both global and regional facial areas, the proposed hierarchical network with LSTM structure can effectively improve the diagnostic accuracy and extract paralysis detail from the low-level shape, contour to sematic level features. By segmenting the facial area into two palsy regions, the proposed method can discriminate FNP from normal face accurately and significantly reduce the effect caused by age wrinkles and unrepresentative organs with shape and position variations on feature learning. Experiment on the YouTube Facial Palsy Database and Extended CohnKanade Database shows that the proposed method is superior to the state of the art deep learning methods.