Simulation of dynamic orofacial movements using a constitutive law varying with muscle activation

Finite element analysis of biomechanical interactions of a subcutaneous suspension suture and human face soft-tissue: a cadaver study

In order to study the local interactions between facial soft-tissues and a Silhouette Soft® suspension suture, a CE marked medical device designed for the repositioning of soft tissues in the face and the neck, Finite element simulations were run, in which a model of the suture was embedded in a three-layer Finite Element structure that accounts for the local mechanical organization of human facial soft tissues. A 2D axisymmetric model of the local interactions was designed in ANSYS, in which the geometry of the tissue, the boundary conditions and the applied loadings were considered to locally mimic those of human face soft tissue constrained by the suture in facial tissue repositioning. The Silhouette Soft suture is composed of a knotted thread and sliding cones that are anchored in the tissue. Hence, simulating these interactions requires special attention for an accurate modelling of contact mechanics. As tissue is modelled as a hyper-elastic material, the displacement of the facial soft tissue changes in a nonlinear way with the intensity of stress induced by the suture and the number of the cones. Our simulations show that for a 4-cone suture a displacement of 4.35 mm for a 2.0 N external loading and of 7.6 mm for 4.0 N. Increasing the number of cones led to the decrease in the equivalent local strain (around 20%) and stress (around 60%) applied to the tissue. The simulated displacements are in general agreement with experimental observations.

Reinforcement learning coupled with finite element modeling for facial motion learning

BACKGROUND AND OBJECTIVE

Facial palsy patients or patients with facial transplantation have abnormal facial motion due to altered facial muscle functions and nerve damage. Computer-aided system and physics-based models have been developed to provide objective and quantitative information. However, the predictive capacity of these solutions is still limited to explore the facial motion patterns with emerging properties. The present study aims to couple the reinforcement learning and the finite element modeling for facial motion learning and prediction.

METHODS

A novel modeling workflow for learning facial motion was developed. A physically-based model of the face within the Artisynth modeling platform was used. Information exchange protocol was proposed to link reinforcement learning and rigid multi-bodies dynamics outcomes. Two reinforcement learning algorithms (deep deterministic policy gradient (DDPG) and Twin-delayed DDPG (TD3)) were used and implemented to drive the simulations of symmetry-oriented and smile movements. Numerical outcomes were compared to experimental observations (Bosphorus database) for evaluation and validation purposes.

RESULTS

As result, after more than 100 episodes of exploring the environment, the agent starts to learn from previous trials and can find the optimal policy after more than 300 episodes of training. Regarding the symmetry-oriented motion, the muscle excitations predicted by the trained agent help to increase the value of reward from R = -2.06 to R = -0.23, which counts for ∼89% improvement of the symmetry value of the face. For smile-oriented motion, two points at the edge of the mouth move up 0.35 cm, which is within the range of movements estimated from the Bosphorus database (0.4 ± 0.32 cm).

CONCLUSIONS

The present study explored the muscle excitation patterns by coupling reinforcement learning with a detailed finite element model of the face. We developed, for the first time, a novel coupling scheme to integrate the finite element simulation into the reinforcement learning process for facial motion learning. As perspectives, this present workflow will be applied for facial palsy and facial transplantation patients to guide and optimize the functional rehabilitation program.

A novel incremental simulation of facial changes following orthognathic surgery using FEM with realistic lip sliding effect

Accurate prediction of facial soft-tissue changes following orthognathic surgery is crucial for surgical outcome improvement. We developed a novel incremental simulation approach using finite element method (FEM) with a realistic lip sliding effect to improve the prediction accuracy in the lip region. First, a lip-detailed mesh is generated based on accurately digitized lip surface points. Second, an improved facial soft-tissue change simulation method is developed by applying a lip sliding effect along with the mucosa sliding effect. Finally, the orthognathic surgery initiated soft-tissue change is simulated incrementally to facilitate a natural transition of the facial change and improve the effectiveness of the sliding effects. Our method was quantitatively validated using 35 retrospective clinical data sets by comparing it to the traditional FEM simulation method and the FEM simulation method with mucosa sliding effect only. The surface deviation error of our method showed significant improvement in the upper and lower lips over the other two prior methods. In addition, the evaluation results using our lip-shape analysis, which reflects clinician's qualitative evaluation, also proved significant improvement of the lip prediction accuracy of our method for the lower lip and both upper and lower lips as a whole compared to the other two methods. In conclusion, the prediction accuracy in the clinically critical region, i.e., the lips, significantly improved after applying incremental simulation with realistic lip sliding effect compared with the FEM simulation methods without the lip sliding effect.

Quantal biomechanical effects in speech postures of the lips

The unique biomechanical and functional constraints on human speech make it a promising area for research investigating modular control of movement. The present article illustrates how a modular control approach to speech can provide insights relevant to understanding both motor control and observed variation across languages. We specifically explore the robust typological finding that languages produce different degrees of labial constriction using distinct muscle groupings and concomitantly distinct lip postures. Research has suggested that these lip postures exploit biomechanical regions of nonlinearity between neural activation and movement, also known as quantal regions, to allow movement goals to be realized despite variable activation signals. We present two sets of computer simulations showing that these labial postures can be generated under the assumption of modular control and that the corresponding modules are biomechanically robust: first to variation in the activation levels of participating muscles, and second to interference from surrounding muscles. These results provide support for the hypothesis that biomechanical robustness is an important factor in selecting the muscle groupings used for speech movements and provide insight into the neurological control of speech movements and how biomechanical and functional constraints govern the emergence of speech motor modules. We anticipate that future experimental work guided by biomechanical simulation results will provide new insights into the neural organization of speech movements.NEW & NOTEWORTHY This article provides additional evidence that speech motor control is organized in a modular fashion and that biomechanics constrain the kinds of motor modules that may emerge. It also suggests that speech can be a fruitful domain for the study of modularity and that a better understanding of speech motor modules will be useful for speech research. Finally, it suggests that biomechanical modeling can serve as a useful complement to experimental work when studying modularity.

Human sound systems are shaped by post-Neolithic changes in bite configuration

Linguistic diversity, now and in the past, is widely regarded to be independent of biological changes that took place after the emergence of Homo sapiens We show converging evidence from paleoanthropology, speech biomechanics, ethnography, and historical linguistics that labiodental sounds (such as "f" and "v") were innovated after the Neolithic. Changes in diet attributable to food-processing technologies modified the human bite from an edge-to-edge configuration to one that preserves adolescent overbite and overjet into adulthood. This change favored the emergence and maintenance of labiodentals. Our findings suggest that language is shaped not only by the contingencies of its history, but also by culturally induced changes in human biology.

Intravelar and Extravelar Portions of Soft Palate Muscles in Velic Constrictions: A Three-Dimensional Modeling Study

Purpose This study predicts and simulates the function and relative contributions of the intravelar and extravelar portions of the levator veli palatini (LVP) and palatoglossus (PG) muscles in velic constrictions. Method A finite element-based model of the 3-dimensional upper airway structures (palate, pharynx, tongue, jaw, maxilla) was implemented, with LVP and PG divided into intravelar and extravelar portions. Simulations were run to investigate the contributions of these muscles in velopharyngeal port (VPP) closure and constriction of the oropharyngeal isthmus (OPI). Results Simulations reveal that the extravelar portion of LVP, though crucial for lifting the palate, is not sufficient to effect VPP closure. Specifically, the characteristic "bulge" appearing in the posterior soft palate during VPP closure ( Pigott, 1969 ; Serrurier & Badin, 2008 ) is found to result from activation of the intravelar portion of LVP. Likewise, the intravelar portion of posterior PG is crucial in bending the "veil" or "traverse" ( Gick, Francis, Klenin, Mizrahi, & Tom, 2013 ) of the velum anteriorly to produce uvular constrictions of the OPI ( Gick et al., 2014 ). Conclusions Simulations support the view that intravelar LVP and PG play significant roles in VPP and OPI constrictions.

A Systematic Review of Continuum Modeling of Skeletal Muscles: Current Trends, Limitations, and Recommendations

Finite elasticity theory has been commonly used to model skeletal muscle. A very large range of heterogeneous constitutive laws has been proposed. In this review, the most widely used continuum models of skeletal muscles were synthetized and discussed. Trends and limitations of these laws were highlighted to propose new recommendations for future researches. A systematic review process was performed using two reliable search engines as PubMed and ScienceDirect. 40 representative studies (13 passive muscle materials and 27 active muscle materials) were included into this review. Note that exclusion criteria include tendon models, analytical models, 1D geometrical models, supplement papers, and indexed conference papers. Trends of current skeletal muscle modeling relate to 3D accurate muscle representation, parameter identification in passive muscle modeling, and the integration of coupled biophysical phenomena. Parameter identification for active materials, assumed fiber distribution, data assumption, and model validation are current drawbacks. New recommendations deal with the incorporation of multimodal data derived from medical imaging, the integration of more biophysical phenomena, and model reproducibility. Accounting for data uncertainty in skeletal muscle modeling will be also a challenging issue. This review provides, for the first time, a holistic view of current continuum models of skeletal muscles to identify potential gaps of current models according to the physiology of skeletal muscle. This opens new avenues for improving skeletal muscle modeling in the framework of in silico medicine.

IMAGE-BASED SKELETAL MUSCLE COORDINATION: CASE STUDY ON A SUBJECT SPECIFIC FACIAL MIMIC SIMULATION

Facial muscle coordination is a fundamental mechanism for facial mimics and expressions. The understanding of this complex mechanism leads to better diagnosis and treatment of facial disorders like facial palsy or disfigurement. The objective of this work was to use magnetic resonance imaging (MRI) technique to characterize the activation behavior of facial muscles and then simulate their coordination mechanism using a subject specific finite element model. MRI data of lower head of a healthy subject were acquired in neutral and in the pronunciation of the sound [o] positions. Then, a finite element model was derived directly from acquired MRI images in neutral position. Transversely-isotropic, hyperelastic, quasi-incompressible behavior law was implemented for modeling facial muscles. The simulation to produce the pronunciation of the sound [o] was performed by the cumulative coordination between three pairs of facial mimic muscles (Zygomaticus Major (ZM), Levator Labii Superioris (LLS), Levator Anguli Oris (LAO)). Mean displacement amplitude showed a good agreement with a relative deviation of 15% between numerical outcome and MRI-based measurement when all three muscles are involved. This study elucidates, for the first time, the facial muscle coordination using in vivo data leading to improve the model understanding and simulation outcomes.

sEMG-assisted inverse modelling of 3D lip movement: a feasibility study towards person-specific modelling

We propose a surface-electromyographic (sEMG) assisted inverse-modelling (IM) approach for a biomechanical model of the face to obtain realistic person-specific muscle activations (MA) by tracking movements as well as innervation trajectories. We obtained sEMG data of facial muscles and 3D positions of lip markers in six volunteers and, using a generic finite element (FE) face model in ArtiSynth, performed inverse static optimisation with and without sEMG tracking on both simulation data and experimental data. IM with simulated data and experimental data without sEMG data showed good correlations of tracked positions (0.93 and 0.67) and poor correlations of MA (0.27 and 0.20). When utilising the sEMG-assisted IM approach, MA correlations increased drastically (0.83 and 0.59) without sacrificing performance in position correlations (0.92 and 0.70). RMS errors show similar trends with an error of 0.15 in MA and of 1.10 mm in position. Therefore, we conclude that we were able to demonstrate the feasibility of an sEMG-assisted inverse modelling algorithm for the perioral region. This approach may help to solve the ambiguity problem in inverse modelling and may be useful, for instance, in future applications for preoperatively predicting treatment-related function loss.

Simulation of facial expressions using person-specific sEMG signals controlling a biomechanical face model

PURPOSE

Functional inoperability in advanced oral cancer is difficult to assess preoperatively. To assess functions of lips and tongue, biomechanical models are required. Apart from adjusting generic models to individual anatomy, muscle activation patterns (MAPs) driving patient-specific functional movements are necessary to predict remaining functional outcome. We aim to evaluate how volunteer-specific MAPs derived from surface electromyographic (sEMG) signals control a biomechanical face model.

METHODS

Muscle activity of seven facial muscles in six volunteers was measured bilaterally with sEMG. A triple camera set-up recorded 3D lip movement. The generic face model in ArtiSynth was adapted to our needs. We controlled the model using the volunteer-specific MAPs. Three activation strategies were tested: activating all muscles [Formula: see text], selecting the three muscles showing highest muscle activity bilaterally [Formula: see text]-this was calculated by taking the mean of left and right muscles and then selecting the three with highest variance-and activating the muscles considered most relevant per instruction [Formula: see text], bilaterally. The model's lip movement was compared to the actual lip movement performed by the volunteers, using 3D correlation coefficients [Formula: see text].

RESULTS

The correlation coefficient between simulations and measurements with [Formula: see text] resulted in a median [Formula: see text] of 0.77. [Formula: see text] had a median [Formula: see text] of 0.78, whereas with [Formula: see text] the median [Formula: see text] decreased to 0.45.

CONCLUSION

We demonstrated that MAPs derived from noninvasive sEMG measurements can control movement of the lips in a generic finite element face model with a median [Formula: see text] of 0.78. Ultimately, this is important to show the patient-specific residual movement using the patient's own MAPs. When the required treatment tools and personalisation techniques for geometry and anatomy become available, this may enable surgeons to test the functional results of wedge excisions for lip cancer in a virtual environment and to weigh surgery versus organ-sparing radiotherapy or photodynamic therapy.

MRI-based finite element modeling of facial mimics: a case study on the paired zygomaticus major muscles

Finite element simulation of facial mimics provides objective indicators about soft tissue functions for improving diagnosis, treatment and follow-up of facial disorders. There is a lack of in vivo experimental data for model development and validation. In this study, the contribution of the paired Zygomaticus Major (ZM) muscle contraction on the facial mimics was investigated using in vivo experimental data derived from MRI. Maximal relative differences of 7.7% and 37% were noted between MRI-based measurements and numerical outcomes for ZM and skin deformation behaviors respectively. This study opens a new direction to simulate facial mimics with in vivo data.

The Quantal Larynx: The Stable Regions of Laryngeal Biomechanics and Implications for Speech Production

PURPOSE

Recent proposals suggest that (a) the high dimensionality of speech motor control may be reduced via modular neuromuscular organization that takes advantage of intrinsic biomechanical regions of stability and (b) computational modeling provides a means to study whether and how such modularization works. In this study, the focus is on the larynx, a structure that is fundamental to speech production because of its role in phonation and numerous articulatory functions.

METHOD

A 3-dimensional model of the larynx was created using the ArtiSynth platform (http://www.artisynth.org). This model was used to simulate laryngeal articulatory states, including inspiration, glottal fricative, modal prephonation, plain glottal stop, vocal-ventricular stop, and aryepiglotto-epiglottal stop and fricative.

RESULTS

Speech-relevant laryngeal biomechanics is rich with "quantal" or highly stable regions within muscle activation space.

CONCLUSIONS

Quantal laryngeal biomechanics complement a modular view of speech control and have implications for the articulatory-biomechanical grounding of numerous phonetic and phonological phenomena.

Analysis of shear wave propagation derived from MR elastography in 3D thigh skeletal muscle using subject specific finite element model

The purpose of this study was to develop a subject specific finite element model derived from MRI images to numerically analyze the MRE (magnetic resonance elastography) shear wave propagation within skeletal thigh muscles. A sagittal T2 CUBE MRI sequence was performed on the 20-cm thigh segment of a healthy male subject. Skin, adipose tissue, femoral bone and 11 muscles were manually segmented in order to have 3D smoothed solid and meshed models. These tissues were modeled with different constitutive laws. A transient modal dynamics analysis was applied to simulate the shear wave propagation within the thigh tissues. The effects of MRE experimental parameters (frequency, force) and the muscle material properties (shear modulus: C10) were analyzed through the simulated shear wave displacement within the vastus medialis muscle. The results showed a plausible range of frequencies (from 90Hz to 120 Hz), which could be used for MRE muscle protocol. The wave amplitude increased with the level of the force, revealing the importance of the boundary condition. Moreover, different shear displacement patterns were obtained as a function of the muscle mechanical properties. The present study is the first to analyze the shear wave propagation in skeletal muscles using a 3D subject specific finite element model. This study could be of great value to assist the experimenters in the set-up of MRE protocols.

Subject-Specific Biomechanical Modelling of the Oropharynx: Towards Speech Production

Biomechanical models of the oropharynx are beneficial to treatment planning of speech impediments by providing valuable insight into the speech function such as motor control. In this paper, we develop a subject-specific model of the oropharynx and investigate its utility in speech production. Our approach adapts a generic tongue-jaw-hyoid model (Stavness et al. 2011) to fit and track dynamic volumetric MRI data of a normal speaker, subsequently coupled to a source-filter based acoustic synthesizer. We demonstrate our model's ability to track tongue tissue motion, simulate plausible muscle activation patterns, as well as generate acoustic results that have comparable spectral features to the associated recorded audio. Finally, we propose a method to adjust the spatial resolution of our subject-specific tongue model to match the fidelity level of our MRI data and speech synthesizer. Our findings suggest that a higher resolution tongue model - using similar muscle fibre definition - does not show a significant improvement in acoustic performance, for our speech utterance and at this level of fidelity; however we believe that our approach enables further refinements of the muscle fibres suitable for studying longer speech sequences and finer muscle innervation using higher resolution dynamic data.

Generating Facial Expressions Using an Anatomically Accurate Biomechanical Model

This paper presents a computational framework for modelling the biomechanics of human facial expressions. A detailed high-order (Cubic-Hermite) finite element model of the human head was constructed using anatomical data segmented from magnetic resonance images. The model includes a superficial soft-tissue continuum consisting of skin, the subcutaneous layer and the superficial Musculo-Aponeurotic system. Embedded within this continuum mesh, are 20 pairs of facial muscles which drive facial expressions. These muscles were treated as transversely-isotropic and their anatomical geometries and fibre orientations were accurately depicted. In order to capture the relative composition of muscles and fat, material heterogeneity was also introduced into the model. Complex contact interactions between the lips, eyelids, and between superficial soft tissue continuum and deep rigid skeletal bones were also computed. In addition, this paper investigates the impact of incorporating material heterogeneity and contact interactions, which are often neglected in similar studies. Four facial expressions were simulated using the developed model and the results were compared with surface data obtained from a 3D structured-light scanner. Predicted expressions showed good agreement with the experimental data.

The distributed lambda (λ) model (DLM): a 3-D, finite-element muscle model based on Feldman's λ model; assessment of orofacial gestures

PURPOSE

The authors aimed to design a distributed lambda model (DLM), which is well adapted to implement three-dimensional (3-D), finite-element descriptions of muscles.

METHOD

A muscle element model was designed. Its stress-strain relationships included the active force-length characteristics of the λ model along the muscle fibers, together with the passive properties of muscle tissues in the 3-D space. The muscle element was first assessed using simple geometrical representations of muscles in the form of rectangular bars. It was then included in a 3-D face model, and its impact on lip protrusion was compared with the impact of a Hill-type muscle model.

RESULTS

The force-length characteristic associated with the muscle elements matched well with the invariant characteristics of the λ model. The impact of the passive properties was assessed. Isometric force variation and isotonic displacements were modeled. The comparison with a Hill-type model revealed strong similarities in terms of global stress and strain.

CONCLUSION

The DLM accounted for the characteristics of the λ model. Biomechanically, no clear differences were found between the DLM and a Hill-type model. Accurate evaluations of the λ model, based on the comparison between data and simulations, are now possible with 3-D biomechanical descriptions of the speech articulators because of the DLM.

A finite element model of the face including an orthotropic skin model under in vivo tension

Computer models of the human face have the potential to be used as powerful tools in surgery simulation and animation development applications. While existing models accurately represent various anatomical features of the face, the representation of the skin and soft tissues is very simplified. A computer model of the face is proposed in which the skin is represented by an orthotropic hyperelastic constitutive model. The in vivo tension inherent in skin is also represented in the model. The model was tested by simulating several facial expressions by activating appropriate orofacial and jaw muscles. Previous experiments calculated the change in orientation of the long axis of elliptical wounds on patients' faces for wide opening of the mouth and an open-mouth smile (both 30(o)). These results were compared with the average change of maximum principal stress direction in the skin calculated in the face model for wide opening of the mouth (18(o)) and an open-mouth smile (25(o)). The displacements of landmarks on the face for four facial expressions were compared with experimental measurements in the literature. The corner of the mouth in the model experienced the largest displacement for each facial expression (∼11-14 mm). The simulated landmark displacements were within a standard deviation of the measured displacements. Increasing the skin stiffness and skin tension generally resulted in a reduction in landmark displacements upon facial expression.

On modelling large deformations of heterogeneous biological tissues using a mixed finite element formulation

This study addresses the issue of modelling material heterogeneity of incompressible bodies. It is seen that when using a mixed (displacement-pressure) finite element formulation, the basis functions used for pressure field may not be able to capture the nonlinearity of material parameters, resulting in pseudo-residual stresses. This problem can be resolved by modifying the constitutive relation using Flory's decomposition of the deformation gradient. A two-parameter Mooney-Rivlin constitutive relation is used to demonstrate the methodology. It is shown that for incompressible materials, the modification does not alter the mechanical behaviour described by the original constitutive model. In fact, the modified constitutive equation shows a better predictability when compared against analytical solutions. Two strategies of describing the material variation (i.e. linear and step change) are explained, and their solutions are evaluated for an ideal two-material interfacing problem. When compared with the standard tied coupling approach, the step change method exhibited a much better agreement because of its ability to capture abrupt changes of the material properties. The modified equation in conjunction with integration point-based material heterogeneity is then used to simulate the deformations of heterogeneous biological structures to illustrate its applications.

A biomechanical modeling study of the effects of the orbicularis oris muscle and jaw posture on lip shape

PURPOSE

The authors' general aim is to use biomechanical models of speech articulators to explore how possible variations in anatomical structure contribute to differences in articulatory strategies and phone systems across human populations. Specifically, they investigated 2 issues: (a) the link between lip muscle anatomy and variability in lip gestures and (b) the constraints of coupled lip/jaw biomechanics on jaw posture in labial sounds.

METHOD

The authors used a model coupling the jaw, tongue, and face. First, the influence of the orbicularis oris (OO) anatomical implementation was analyzed by assessing how changes in depth (from epidermis to the skull) and peripheralness (proximity to the lip horn center) affected lip shaping. Second, the capability of the lip/jaw system to generate protrusion and rounding, or labial closure, was evaluated for different jaw heights.

RESULTS

Results showed that a peripheral and moderately deep OO implementation is most appropriate for protrusion and rounding; a superficial implementation facilitates closure; protrusion and rounding require a high jaw position; and closure is achievable for various jaw heights.

CONCLUSIONS

Models provide objective information regarding possible links between anatomical and speech production variability across humans. Comparisons with experimental data will illustrate how motor control and cultural factors cope with these constraints.

Facial mimics simulation using MRI and finite element analysis

Recent research studies aimed to simulate facial expressions or motions due to muscle contraction using biomechanical models ranging from basic to advanced muscle constitutive models. However, these models are based on generic geometries and there is a lack of experimental data for the numerical simulation as well as for the model validation in a clinical context. The objective of our present study was to perform facial mimics simulation using subject specific data derived from MRI technique. Zygomaticus major muscle is modelled as a transversely isotropic hyperelastic material. Then the resulting effect of its shortening and lengthening process on the facial mimics simulation was performed using Finite Element Analysis. Simulation results were presented and discussed. Such study will be of interest for defining objective criteria to evaluate the facial disfigurement patients and to perform the functional rehabilitation.