A new method for automatic tracking of facial landmarks in 3D motion captured images (4D)

Ultra-dense Motion Capture: An exploratory full-automatic approach for dense tracking of breast motion in 4D

Understanding the dynamic deformation pattern and biomechanical properties of breasts is crucial in various fields, including designing ergonomic bras and customized prostheses, as well as in clinical practice. Previous studies have recorded and analyzed the dynamic behaviors of the breast surface using 4D scanning, which provides a sequence of 3D meshes during movement with high spatial and temporal resolutions. However, these studies are limited by the lack of robust and automated data processing methods which result in limited data coverage or error-prone analysis results. To address this issue, we identify revealing inter-frame dense correspondence as the core challenge towards conducting reliable and consistent analysis of the 4D scanning data. We proposed a fully-automatic approach named Ulta-dense Motion Capture (UdMC) using Thin-plate Spline (TPS) to augment the sparse landmarks recorded via motion capture (MoCap) as initial dense correspondence and then rectified it with a sophisticated post-alignment scheme. Two downstream tasks are demonstrated to validate its applicability: virtual landmark tracking and deformation intensity analysis. For evaluation, a dynamic 4D human breast anthropometric dataset DynaBreastLite was constructed. The results show that our approach can robustly capture the dynamic deformation characteristics of the breast surfaces, significantly outperforms baselines adapted from previous works in terms of accuracy, consistency, and efficiency. For 10 fps dataset, average error of 0.25 cm on control-landmarks and 0.33 cm on non-control (arbitrary) landmarks were achieved, with 17-70 times faster computation time. Evaluation was also carried out on 60 fps and 120 fps datasets, with consistent and large performance gaining being observed. The proposed method may contribute to advancing research in breast anthropometry, biomechanics, and ergonomics by enabling more accurate tracking of the breast surface deformation patterns and dynamic characteristics.

Frontiers in Three-Dimensional Surface Imaging Systems for 3D Face Acquisition in Craniofacial Research and Practice: An Updated Literature Review

Digitalizing all aspects of dental care is a contemporary approach to ensuring the best possible clinical outcomes. Ongoing advancements in 3D face acquisition have been driven by continuous research on craniofacial structures and treatment effects. An array of 3D surface-imaging systems are currently available for generating photorealistic 3D facial images. However, choosing a purpose-specific system is challenging for clinicians due to variations in accuracy, reliability, resolution, and portability. Therefore, this review aims to provide clinicians and researchers with an overview of currently used or potential 3D surface imaging technologies and systems for 3D face acquisition in craniofacial research and daily practice. Through a comprehensive literature search, 71 articles meeting the inclusion criteria were included in the qualitative analysis, investigating the hardware, software, and operational aspects of these systems. The review offers updated information on 3D surface imaging technologies and systems to guide clinicians in selecting an optimal 3D face acquisition system. While some of these systems have already been implemented in clinical settings, others hold promise. Furthermore, driven by technological advances, novel devices will become cost-effective and portable, and will also enable accurate quantitative assessments, rapid treatment simulations, and improved outcomes.

Comparison of multiple 3D scanners to capture foot, ankle, and lower leg morphology

BACKGROUND

3D scanning of the foot and ankle is gaining popularity as an alternative method to traditional plaster casting to fabricate ankle-foot orthoses (AFOs). However, comparisons between different types of 3D scanners are limited.

OBJECTIVES

The aim of this study was to evaluate the accuracy and speed of seven 3D scanners to capture foot, ankle, and lower leg morphology to fabricate AFOs.

STUDY DESIGN

Repeated-measures design.

METHODS

The lower leg region of 10 healthy participants (mean age 27.8 years, standard deviation [SD] 9.3) was assessed with 7 different 3D scanners: Artec Eva (Eva), Structure Sensor (SS I), Structure Sensor Mark II (SS II), Sense 3D Scanner (Sense), Vorum Spectra (Spectra), Trnio 3D Scanner App on iPhone 11 (Trnio 11), and Trnio 3D Scanner App on iPhone 12 (Trnio 12). The reliability of the measurement protocol was confirmed initially. The accuracy was calculated by comparing the digital scan with clinical measures. A percentage difference of #5% was considered acceptable. Bland and Altman plots were used to show the mean bias and limit of agreement (LoA) for each 3D scanner. Speed was the time needed for 1 complete scan.

RESULTS

The mean accuracy ranged from 6.4% (SD 10.0) to 230.8% (SD 8.4), with the SS I (21.1%, SD 6.8), SS II (21.7%, SD 7.5), and Eva (2.5%, SD 4.5) within an acceptable range. Similarly, Bland and Altman plots for Eva, SS I, and SS II showed the smallest mean bias and LoA 21.7 mm (LoA 25.8 to 9.3), 21.0 mm (LoA 210.3 to 8.3), and 0.7 mm (LoA 213 to 11.5), respectively. The mean speed of the 3D scanners ranged from 20.8 seconds (SD 8.1, SS I) to 329.6 seconds (SD 200.2, Spectra).

CONCLUSIONS

Eva, SS I, and SS II appear to be the most accurate and fastest 3D scanners for capturing foot, ankle, and lower leg morphology, which could be used for AFO fabrication.

Assessment of the reproducibility and accuracy of the Visia® Complexion Analysis Camera System for objective skin analysis of facial wrinkles and skin age

Objective

This study aimed to investigate the reproducibility and accuracy of the Visia® Complexion Analysis Camera System by Canfield Scientific for objective skin analysis.

Methods

Nineteen participants underwent facial capture with the Visia® camera following a standardised protocol. During the first session, the participants sat down and positioned their faces in a capture rig, closed their eyes and had their faces captured from the left, front and right sides, with threefold repetition of the captures from the front side. After 4 weeks, the participants underwent recapture in a similar manner. Based on the frontal views, data for two measurement methods of the Visia® camera system, the absolute scores and the percentiles, were obtained with regard to the skin criterion wrinkles via automated software calculation. Means and standard deviations were evaluated. Based on the side views, the data for the Truskin Ages® were calculated by the Visia® camera system and compared with the calendrical ages, which served as the gold standard for comparison.

Results

In the assessment of the reproducibility of the data of the capture system the standard deviation from the frontal captures among all participants was about 3% when the absolute scores of the wrinkles were compared with each other; specifically, the average deviation was 3.36% during the first capture session and 3.4% during the second capture session. Meanwhile, the standard deviation of the measurements was about 9% when the percentiles were compared; specifically, the average deviation was 8.2% during the first capture session and 10.7% during the second capture session. In the assessment of the accuracy the correlation between the calendrical age and the calculated Truskin Age® for both facial sides was very high at a correlation coefficient rho value of >0.8 (right side: r=0.896; left side: r=0.827) and statistically significant at a p-value of <0.001. The average calendrical age and Truskin Age® deviated only slightly from each other and did not differ significantly (right side: p=0.174; left side: p=0.190). The Truskin Age® was slightly higher than the calendrical age by a mean value of 1.37 years for both facial sides. The analysis of the absolute differences revealed that in 50% of the cases, there was a maximum difference of 3 years, and in 75% of the cases, there were maximum differences of 4.5 years for the right side and 5.5 years for the left side.

Conclusion

The assessment of the reproducibility and accuracy of the objective measurement method, the Visia® camera system, contributed to the validation of the system. The evaluation of the reproducibility revealed a satisfactory precision of the repeated captures when investigating facial wrinkles. Absolute scores should be preferred over percentiles owing to their better precision. The calculation of the accuracy of the Truskin Age® data from the Visia® camera system revealed only a slight deviation from the true calendrical ages. The correlation between both data groups was highly significant.

An innovative analysis of nasolabial dynamics of surgically managed adult patients with unilateral cleft lip and palate using 3D facial motion capture

AIM

To compare dynamic nasolabial movement between end-of-treatment cleft and a matched non-cleft group in adult patients.

MATERIALS AND METHODS

Thirteen treated adult participants with unilateral cleft lip and palate had images taken using a facial motion capture system performing a maximum smile. Seventeen landmarks were automatically tracked. For each landmark pair, on either side of the midline, changes in the x, y, and z directions were used to analyze the magnitude of displacement and path of motion. An asymmetry score was developed at rest, mid-smile, and maximum smile to assess the shape of the mouth and/or nose.

RESULTS

At maximum smile, displacement of right and left cheilion was clinically and statistically (p < 0.05) less in the cleft group. The lip asymmetry score was greater (p < 0.05) at each time point in the cleft group using the clinical midline. Using Procrustes superimposition, the differences were significant (p < 0.05) only at rest and mid-smile. The alar bases were displaced significantly less (p < 0.05) in the z direction in the cleft group. The asymmetry score of the alar base was significantly higher using the clinical midline than using Procrustes superimposition in patients with cleft conditions (p < 0.001). In the cleft group, at maximum smile, the right and left cristae philter moved significantly less (p < 0.05) in the x and z directions.

CONCLUSIONS

There was an increase in asymmetry score of the corners of the mouth and alar bases from rest to maximum smile. The lips were similar in shape but oriented differently in the faces of patients with cleft conditions than in individuals without those conditions.

Integration accuracy of craniofacial cone-beam computed tomography images with three-dimensional facial scans according to different registration areas

OBJECTIVES

To evaluate the integration accuracy of cone-beam computed tomography (CBCT) images with three-dimensional (3D) facial scans according to different registration areas.

MATERIALS AND METHODS

Twenty-five patients (14 males and 11 females), with a mean age of 19.0 ± 11.3 years, were included in this study. Each patient underwent CBCT and facial scans on the same day in an upright position. The facial scans were integrated with the corresponding soft-tissue images of CBCT scans. Three methods were used to integrate the two imaging modalities based on the facial regions scanned: R1, only the forehead and nasal bridge area were included; R2, the right and left malar area were included; and R3, the forehead, nasal bridge, and malar areas were included. The integration accuracy between the facial scans and CBCT images was evaluated by color-mapping methods and average surface distances, calculated by measuring the 3D distances between the surface points on the two superimposed images.

RESULTS

The average surface differences between facial scans and CBCT images were less than 1.0 mm in all three methods. The R3 method showed fewer differences between the facial scans and CBCT images than the other methods did.

CONCLUSIONS

Facial scans obtained using a low-cost facial scanner showed clinically acceptable performance. The integration accuracy of facial and CBCT scans can be increased by including the forehead, nasal bridge, and malar areas as registration areas.

Cloner 3D photogrammetric facial scanner: Assessment of accuracy in a controlled clinical study

OBJECTIVE

To evaluate the accuracy of facial measurements on three-dimensional images obtained using a new photogrammetric scanner.

MATERIAL AND METHODS

A total of 11 participants were included in the study. Nine customized adhesive labels were used to identify the facial landmarks: Trichion (Tri), Glabella (G), Right (Exr) and Left (Exl), Pronasal (Pn), Subnasal (Sn), Chelion right (Chr) and left (Chl) and Mentonian (Me). Two trained and calibrated examiners were responsible for performing seven linear measurements for each participant (Tri-G, Sn-Me, Exr-Exl, Chr-Chl, Exr-Chr, Exl-Chl, Pn-Sn) first with a digital caliper and later with a three-dimensional model obtained after digitalization with photogrammetric technology. The intraclass correlation coefficient (ICC), mean difference, SD, and Bland-Altman correlation were used to compare the measurements performed.

RESULTS

Intra and inter-examiner reliability were excellent (ICC >0.9). In general, the measurements presented a variation of a minor 2.0 mm. However, only three measures (Sn-Me, Exr-Exl, and Exr-Chr) were outside the clinical acceptability range.

CONCLUSIONS

The 3D Cloner scanner showed clinically acceptable accuracy comparable to the digital caliper with a variation of -0.8 ± 1.2 mm. Inter- and intra-examiner agreement on digital measurements was also observed.

CLINICAL SIGNIFICANCE

Scanners with accurate 3D model reproductions associated with reliable digital measurements provide a more precise diagnosis and better planning in orofacial treatment.

Accuracy of three-dimensional optical devices for facial soft-tissue measurement in clinical practice of stomatology: A PRISMA systematic review

BACKGROUND

The accuracy of 3-dimensional (3D) optical devices for facial soft-tissue measurement is essential to the success of clinical treatment in stomatology. The aim of the present systematic review was to summarize the accuracy of 3D optical devices used for facial soft-tissue assessment in stomatology.

METHODS

An extensive systematic literature search was performed in the PubMed/MEDLINE, Embase, Scopus and Cochrane Library databases for studies published in the English language up to May 2022 in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-analyses guidelines. Peer-reviewed journal articles evaluating the facial soft-tissue morphology by 3D optical devices were included. The risk of bias was performed using the Quality Assessment Tool for Diagnostic Accuracy Studies-2 guidelines by the 2 reviewers. The potential publication bias was analyzed using the Review Manager software.

RESULTS

The query returned 1853 results. A total of 38 studies were included in this review. Articles were categorized based on the principle of devices: laser-based scanning, structured-light scanning, stereophotogrammetry and red, green, blue-depth camera.

CONCLUSION

Overall, the 3D optical devices demonstrated excellent accuracy and reliability for facial soft-tissue measurement in stomatology. red, green, blue-depth camera can collect accurate static and dynamic 3D facial scans with low cost and high measurement accuracy. Practical needs and availability of resources should be considered when these devices are used in clinical settings.

Anthropometric Comparison of 3-Dimensional Facial Scan Taken With a Low-Cost Facial Scanner With Cone-Beam Computed Tomography Scan

OBJECTIVE

The aim of this study was to evaluate the accuracy of 3-dimensional (3D) facial scans taken with a low-cost facial scanner compared with cone-beam computed tomography (CBCT).

MATERIALS AND METHODS

Twenty-five patients had their CBCT scan and 3D facial scan taken on the same day. Facial scans were taken with a low-cost facial scanner (Bellus3D, ARC-1 scanner) in an upright position. The facial scans were imported into Invivo5 software and were integrated with the corresponding CBCT scans. Then, 16 soft tissue landmarks were identified on the CBCT scans and the 3D-integrated facial scans. The 6 vertical, 7 horizontal, 10 oblique linear, and 11 angular anthropometric measurements were obtained and compared between the CBCT scans and 3D-integrated facial scans using paired t test and Bland-Altman plots.

RESULTS

The differences between CBCT scans and 3D-integrated facial scans showed no statistical significance except for 2 vertical measurements. Bland-Altman plots showed that all anthropometric measurements were within the limit of agreement. The differences between CBCT scans and 3D-integrated facial scans were <1.5 mm or 1.5 degrees except for 2 vertical measurements.

CONCLUSIONS

Three-dimensional facial scans taken with facial scanner showed a clinical acceptance when compared with CBCT scans.

"Introduction of a low-cost and automated four-dimensional assessment system of the face."

SUMMARY

Existing automated objective grading systems either fail to consider the face's complex 3D morphology or suffer from poor feasibility and usability. Consumer-based Red Green Blue Depth (RGB-D) sensors and/or smartphone integrated 3D hardware can inexpensively collect detailed four-dimensional facial data in real-time but are yet to be incorporated into a practical system. This study aims to evaluate the feasibility of a proof-of-concept automated 4D facial assessment system using an RGB-D sensor (termed OpenFAS) for use in a standard clinical environment. This study was performed on normal adult volunteers and patients with facial nerve palsy (FNP). The setup consists of the Intel RealSense SR300 connected to a laptop running the OpenFAS application. The subject sequentially mimics the facial expressions shown on screen. Each frame is landmarked, and automatic anthropometric calculations are performed. Any errors during each session were noted. Landmarking accuracy was estimated by comparing the 'ground-truth position' of landmarks annotated manually to those placed automatically. 18 participants were included in the study, nine healthy participants and nine patients with FNP. Each session was standardized at approximately 106 seconds. 61.8% of landmarks were automatically annotated within approximately 1.575mm of their ground-truth locations. Our findings support that OpenFAS is usable and feasible in routine settings, laying down the critical groundwork for a facial assessment system that addresses the shortcomings of existing tools. However, the iteration of OpenFAS presented in this study is undoubtedly nascent with future work including improvements to landmarking accuracy, analyses components, and RGB-D technology required before clinical application.

Landmark movement trajectory: A simple reference to understand the dominant muscle contraction in a dynamic smile expression

Insight into the roles of perioral muscles play in smiling is essential to the relevant esthetic and reconstructive treatment. This study analyzed specific landmarks' movement trajectories to understand the dominant muscle activity in smiling. Sixty volunteers (aged 18-45 years) were included, and standardized frontal view images were obtained by three-dimensional photogrammetry. Each image sequence frame was referenced and superimposed onto the neutral image. Six surface landmarks were digitized on each frame to form the entire movement trajectories from resting to maximal smile. The perioral muscles were divided into different groups according to the inserting position of muscular fibers and the contracting direction. The movement paths of landmarks were analyzed to find the major muscle groups involved. The dominant muscle groups throughout smiling were successfully identified by landmark trajectory analysis in all 60 volunteers. Zygomaticus major was the dominant force for the beginning of most smiles. Apart from the zygomaticus major, other perioral muscles played a dominant role throughout smile expression, including levator anguli oris, buccinator and risorius, depressor anguli oris, and the upward muscle group of the upper lip were all identified. Besides, a notable high percentage of smiles presented a mixed pattern, indicating the cooperation of several muscle groups during smiling. Landmark movement trajectory provides a simple reference to understand the dominant perioral muscles in smiling. It can help surgeons get more information for the dynamic treatment of smiles.

Three-Dimensional Motion Capture of a Smile in Repaired Unilateral Cleft Lip: What's Our Vector, Victor?

INTRODUCTION

Following surgical repair, patients with unilateral cleft lip (UCL) exhibit dynamic asymmetry during facial expressions compared to healthy individuals. Previous studies using Euclidean distances to describe this asymmetry fail to take the direction of the movement into account. The aim of this study is to compare differences in participants with UCL and controls using analysis of motion vectors during facial expressions.

METHODS

In this cross-sectional study, twenty-six pediatric participants were recruited: 13 participants with repaired left UCL and 13 participants with no craniofacial diagnosis. Participants were recorded performing a maximal smile by a 4D stereophotogrammetric system. Phases of the smile were divided into closed lip and open lip smiles. Ten regions of interest were analyzed: subnasal area, upper lip, lower lip, oral commissure, and ala on both sides. The motion vectors were calculated and vector magnitude and direction for each region was compared.

RESULTS

Between cleft and control groups, the differences in vector direction were greater than the magnitude differences. Significant differences in vector direction were identified at both oral commissures in the closed lip smile; and at the oral commissure, subnasal, upper lip, and lower lip regions during open lip smile.

CONCLUSIONS

Vector analysis demonstrated significant movement asymmetry during facial animation in participants following UCL repair, not previously identified when analyzing magnitude of skin displacement.

Analysis of Facial Movement in Repaired Unilateral Cleft Lip Using Three-Dimensional Motion Capture

ABSTRACT

Unilateral cleft lip (UCL) is one of the most common craniofacial deformities. Surgical intervention reconstructs lip and nose anatomy; however, some degree of asymmetry persists after repair. This demonstrates a need for a model for studying and improving outcomes for patients with orofacial clefts. This study's main question was whether there is a significant difference in dynamic facial asymmetry between participants with repaired UCLs and control participants during smiling. Ten pediatric subjects with repaired left UCLs and 12 with no craniofacial diagnoses were recorded performing maximum smiles using a markerless 4D video stereophotogrammetrical system. A facial mesh template containing 884 landmarks was conformed to each initial frame and tracked throughout. Kinetic analysis of smiles was performed by calculating landmark 3D Euclidean distance between frames. Patients with left repaired UCL showed increasing facial asymmetry throughout smiling. Oral commissures, upper, and lower lips demonstrated significantly greater movement on the right side (P < 0.05). Control patients showed facial asymmetry during the first half of smiling, with greater movement on the left side. Displacement difference between right and left was significantly greater at oral commissures and upper lips in patients with repaired ULC compared to control patients. This study provides a highly detailed, quantitative analysis of postoperative UCLs, and help improve outcomes of future repair surgeries.

An objective, markerless videosystem for staging facial palsy

Purpose

To propose a new objective, video recording method for the classification of unilateral peripheral facial palsy (UPFP) that relies on mathematical algorithms allowing the software to recognize numerical points on the two sides of the face surface that would be indicative of facial nerve impairment without positioning of markers on the face.

Methods

Patients with UPFP of different House–Brackmann (HB) degrees ranging from II to V were evaluated after video recording during two selected facial movements (forehead frowning and smiling) using a software trained to recognize the face points as numbers. Numerical parameters in millimeters were obtained as indicative values of the shifting of the face points, of the shift differences of the two face sides and the shifting ratio between the healthy (denominator) and the affected side (numerator), i.e., the asymmetry index for the two movements.

Results

For each HB grade, specific asymmetry index ranges were identified with a positive correlation for shift differences and negative correlation for asymmetry indexes.

Conclusions

The use of the present objective system enabled the identification of numerical ranges of asymmetry between the healthy and the affected side that were consistent with the outcome from the subjective methods currently in use.

Kinect-driven Patient-specific Head, Skull, and Muscle Network Modelling for Facial Palsy Patients

BACKGROUND AND OBJECTIVE

Facial palsy negatively affects both professional and personal life qualities of involved patients. Classical facial rehabilitation strategies can recover facial mimics into their normal and symmetrical movements and appearances. However, there is a lack of objective, quantitative, and in-vivo facial texture and muscle activation bio-feedbacks for personalizing rehabilitation programs and diagnosing recovering progresses. Consequently, this study proposed a novel patient-specific modelling method for generating a full patient specific head model from a visual sensor and then computing the facial texture and muscle activation in real-time for further clinical decision making.

METHODS

The modeling workflow includes (1) Kinect-to-head, (2) head-to-skull, and (3) muscle network definition & generation processes. In the Kinect-to-head process, subject-specific data acquired from a new user in neutral mimic were used for generating his/her geometrical head model with facial texture. In particular, a template head model was deformed to optimally fit with high-definition facial points acquired by the Kinect sensor. Moreover, the facial texture was also merged from his/her facial images in left, right, and center points of view. In the head-to-skull process, a generic skull model was deformed so that its shape was statistically fitted with his/her geometrical head model. In the muscle network definition & generation process, a muscle network was defined from the head and skull models for computing muscle strains during facial movements. Muscle insertion points and muscle attachment points were defined as vertex positions on the head model and the skull model respectively based on the standard facial anatomy. Three healthy subjects and two facial palsy patients were selected for validating the proposed method. In neutral positions, magnetic resonance imaging (MRI)-based head and skull models were compared with Kinect-based head and skull models. In mimic positions, infrared depth-based head models in smiling and [u]-pronouncing mimics were compared with appropriate animated Kinect-driven head models. The Hausdorff distance metric was used for these comparisons. Moreover, computed muscle lengths and strains in the tested facial mimics were validated with reported values in literature.

RESULTS

With the current hardware configuration, the patient-specific head model with skull and muscle network could be fast generated within 17.16±0.37s and animated in real-time with the framerate of 40 fps. In neutral positions, the best mean error was 1.91 mm for the head models and 3.21 mm for the skull models. On facial regions, the best mean errors were 1.53 mm and 2.82 mm for head and skull models respectively. On muscle insertion/attachment point regions, the best mean errors were 1.09 mm and 2.16 mm for head and skull models respectively. In mimic positions, these errors were 2.02 mm in smiling mimics and 2.00 mm in [u]-pronouncing mimics for the head models on facial regions. All above error values were computed on a one-time validation procedure. Facial muscles exhibited muscle shortening and muscle elongating for smiling and pronunciation of sound [u] respectively. Extracted muscle features (i.e. muscle length and strain) are in agreement with experimental and literature data.

CONCLUSIONS

This study proposed a novel modeling method for fast generating and animating patient-specific biomechanical head model with facial texture and muscle activation bio-feedbacks. The Kinect-driven muscle strains could be applied for further real-time muscle-oriented facial paralysis grading and other facial analysis applications.

Comparative Study of Three-Dimensional Volume Measurement for Facial Fat Grafting

ABSTRACT

The three-dimensional (3D) volume measurement after facial fat grafting is of great significance to plastic surgeons. It has been ascertained that reliable results rely on the accurate measurement of 3D softwares. Some 3D softwares in literatures have proposed various ways to optimize each step of the procedure, including the pre- and post-operative image acquisition, alignment, calculation, and analysis. Good image alignment between pre- and post-scan is essential to quantify the volumetric change. Once the pre- and post-operative image alignment has slightly bias or deviation, the subsequent volume measurement would also be affected. To our knowledge, 2 types of 3D software have been widely applied in clinic, primarily based on the image-automatically alignment and image-manually alignment. This study aimed to compare the accuracy, repeatability, and reproducibility of the Geomagic Qualify 12.0 software and the 3-Matic 7.0 software in a relatively ideal model of virtual facial fat grafting. A simulated facial fat grafting was first performed for 10 preoperative patients diagnosed with progressive hemifacial atrophy, and the known volumetric change was named as the true value (T value). Then, the facial volumetric change of every case was remeasured 10 times with above 2 kinds of software separately. The mean volumetric change was calculated as Q value and M value. The paired t test, intraclass correlation coefficient, and Bland-Altman analysis showed that the Geomagic Qualify 12.0 software demonstrated a statistically higher accuracy, repeatability, and reproducibility in comparison with the 3-Matic 7.0 software.

A novel method for 3D face symmetry reference plane based on weighted Procrustes analysis algorithm

BACKGROUND

We aimed to establish a novel method, using the weighted Procrustes analysis (WPA) algorithm, which assigns weight to facial anatomical landmarks, to construct a three-dimensional facial symmetry reference plane (SRP) for mandibular deviation patients.

METHODS

Three-dimensional facial SRPs were independently extracted from 15 mandibular deviation patients using both our WPA algorithm and the standard PA algorithm. A reference plane was defined to serve as the ground truth. To determine whether the WPA SRP or the PA SRP was closer to the ground truth, we measured the position error of mirrored landmarks, the facial asymmetry index (FAI) error, and the angle error for the global face and each facial third partition.

RESULTS

The average angle error between the WPA SRP and the ground truth was 1.66 ± 0.81°, which was smaller than that between the PA SRP and the ground truth. The position error of the mirrored landmarks constructed using the WPA algorithm in the global face (3.64 ± 1.53 mm) and each facial partition was lower than that constructed using the PA algorithm. The average FAI error of the WPA SRP was - 7.77 ± 17.02 mm, which was smaller than that of the PA SRP.

CONCLUSIONS

This novel automatic algorithm, based on weighted anatomic landmarks, can provide a more adaptable SRP than the standard PA algorithm when applied to severe mandibular deviation patients and can better simulate the diagnosis strategies of clinical experts.

An Innovative Assessment of the Dynamics of Facial Movements in Surgically Managed Unilateral Cleft Lip and Palate Using 4D Imaging

OBJECTIVE

Assess facial asymmetry during maximum smile in patients with surgically managed unilateral cleft lip and palate (UCLP), using a dynamic 3-dimensional (3D) imaging (4-dimensional) system.

DESIGN

Prospective 2 cohort comparative study.

METHODS

Twenty-five surgically managed UCLP cases and 75 controls at 8 to 10 years of age were recruited. Facial movements during maximum smile were recorded using video stereophotogrammetry at a rate of 60 3D facial images per second. Maximum smile took approximately 3 seconds and generated 180 3D facial images for the analysis. A generic facial mesh which consists of more than 7000 quasi landmarks was used for the assessment of facial asymmetry at 5 key 3D frames representing the pattern of maximum smile.

RESULTS

Statistically significant differences were seen regarding the magnitude of facial asymmetry between the UCLP group and the noncleft controls. Higher average asymmetry in the UCLP group was seen in the 3D frame midway between maximum smile and rest (frame 4) followed by the frame at peak expression of maximum smile (frame 3). The average magnitude of nasolabial asymmetry of the control group was within 0.5 mm in comparison with the UCLP cases which was about 1.8 mm.

CONCLUSION

This study provided for the first time, an objective tool for analysis of the dynamics of muscle movements which provided an unprecedented insight into the anatomical basis of the residual dysmorphology. The research demonstrates the limitations of the primary lip repair in achieving symmetrical results and underpins the required refinements to improve the quality of surgical repair of cleft lip.

Kinematic Analysis of Smiles in the Healthy Pediatric Population Using 3-Dimensional Motion Capture

Introduction:

Facial normalcy, as measured with 2-dimensional or 3-dimensional photographs, has been documented in the healthy pediatric population. However, static images convey far from a complete representation of an individual’s daily interactions with peers. Craniofacial surgery induces changes to soft or osseous tissues and thereby affects dynamic facial expression. To-date, there has not been rigorous, dynamic quantification of normal facial expression. In this study, we used 4-dimensional (4D) imaging to assess the facial expression of healthy children to provide a normative reference point for craniofacial surgeons.

Methods:

A total of 36 healthy pediatric volunteers underwent 4D video recordings while performing a maximal voluntary smile. A face template containing 884 landmarks was registered and tracked throughout the videos using Dimensional Imaging software. Participants were divided into 2 smile groups: open-lip smile and closed-lip smile. Kinematic analysis of smiles was calculated for every landmark from its position in the resting frame to its terminal displacement.

Results:

Composite smiles and Euclidean distance maps were generated displaying areas of greatest displacement near the oral commissures. There was significant difference between closed-lip and open-lip groups in regions of eyes and cheeks. In addition, the open-lip smile group demonstrated significantly greater displacement in the oral commissure on the left side compared to the right ( P < .05); whereas, in the closed-lip group, the eyes and cheeks moved significantly more on the right side.

Conclusion:

This study presents an innovative method that can be used to evaluate facial expressions to help craniofacial surgeons restore functional movement in patients with facial anomalies.

Four-dimensional digital prediction of the esthetic outcome and digital implementation for rehabilitation in the esthetic zone

A technique for 4-dimensional (4D) digital prediction of the outcome of esthetic dentistry for a virtual patient is presented. Static 3D images (which incorporate predicted precise dentition and facial soft tissue in different smiling positions) can be converted into dynamic 3D images by using 3D intraoral imaging, 3D face imaging, and various computer software programs. This strategy can improve the visual perception and quality of esthetic prediction. In addition, the predicted esthetic outcome can be implemented by replicating the contour and shape of digital wax patterns in the definitive ceramic restorations.

Effect of skin tone on the accuracy of hybrid and passive stereophotogrammetry

BACKGROUND

Three-dimensional (3D) surface images acquired from stereophotogrammetry are increasingly being used to plan or evaluate treatment by plastic surgeons. Stereophotogrammetry exists in active, passive, and hybrid forms. Active and hybrid stereophotogrammetry are believed to capture darker surfaces more accurately than passive stereophotogrammetry. The purpose of this study was to investigate whether skin tone has a clinically relevant effect on the accuracy of hybrid and passive stereophotogrammetry.

MATERIALS AND METHODS

Seven subjects with different skin tones were recruited. 3D-printed face and breast were spray-painted in six different colors, ranging from white to black. The skin tones and paint colors were objectified by measuring their melanin index. 3D photos of the subjects and 3D prints were acquired with hybrid and passive stereophotogrammetry. These 3D photos were matched with specialized software, and their geometric differences were calculated.

RESULTS

None of the 3D photos showed a clinically relevant mean inaccuracy. On the 3D prints, hybrid stereophotogrammetry resulted in a smaller standard deviation of the inaccuracies than passive stereophotogrammetry (0.20 ± 0.06 mm vs. 0.35 ± 0.07 mm, p < 0.001). Passive stereophotogrammetry yielded a correlation between the melanin index of the spray paint colors and the standard deviation of the inaccuracy (Pearson's R = 0.60, p = 0.04). On human subjects, no correlation or difference in standard deviation of the accuracy was found.

CONCLUSION

Skin tone does not influence the accuracy of hybrid and passive 3D stereophotogrammetry in a clinically relevant way.

Fluctuating asymmetry of dynamic smiles in normal individuals

The aim of this study was to quantify the fluctuating dynamic facial asymmetry during smiling in a group of 'normal' adults, using three-dimensional (3D) motion facial capture technology. Fifty-four male and 54 female volunteers were recruited. Each subject was imaged using a passive markerless 3D motion capture system (DI4D). Eighteen landmarks were tracked through the 3D capture sequence. A facial asymmetry score was calculated based on either a clinically derived midline or Procrustes alignment; scores were based on the Euclidean distance between landmark pairs. Facial asymmetry scores were determined at three time points: rest, median, and maximum frame. Based on the clinically derived midline and on Procrustes alignment, the differences between male and female volunteers, as well as those at the three different time points, were not clinically significant. However, throughout a smile, facial and lip asymmetry scores increased over the duration of the smile. Fluctuating facial asymmetry exists within individuals, as well as between individuals. Procrustes superimposition and the clinically derived midline produced similar asymmetry scores and both are valid for symmetrical faces. However, with facial asymmetry, Procrustes superimposition may not be a valid measure, and the use of the clinically derived midline may be more appropriate, although this requires further investigation.

Mandibular Advancement Means Lower Facial Enlargement: A 2-Dimensional and 3-Dimensional Analysis

PURPOSE

Bilateral sagittal split osteotomy (BSSO) is frequently used to treat Class II malocclusion for functional and esthetic purposes, with expected changes in the profile. The goal of this study was to assess transversal changes in mandibular advancement by comparing 3-dimensional (3D) photogrammetric modifications and 2-dimensional (2D) radiographic enlargement.

MATERIALS AND METHODS

A cohort study was conducted of patients who underwent an isolated advancement BSSO (Obwegeser-Dal Pont II type) for a Class II malocclusion. All patients had 3D photogrammetric and 2D radiographic evaluations before and after surgery. Frontal cephalograms were used to measure the evolution of bigonial distance (BGD) and 3D photographs were used to measure the evolution of cutaneous BGD (CBGD).

RESULTS

Fourteen patients were included. Mean mandibular advancement was 6 mm. BGD (+6.1 mm; P < 10^-3) and CBGD (+4.2 mm; P = .0017) were significantly increased. The mean ratio of soft tissue response to transversal skeletal changes was 0.81.

CONCLUSION

This 2D and 3D analysis of transversal modifications shows that advancement with the BSSO is responsible for marked lower third facial enlargement. This parameter must be taken into account during the preoperative esthetic assessment to ensure the provision of pertinent information to the patient and the consideration of complementary surgical correction.

Sexual dimorphism in human facial expressions by 3D surface processing

Human face is a dynamic system where facial expressions can rapidly modify geometry of facial features. Facial expressions are believed to be universal across world populations, but only a few studies have explored whether grimacing is sexually dimorphic and if so to what extent. The present paper explores inter- and intra-individual variation of human facial expressions with respect to individual's sex based on a set of neutral and expression-varying 3D facial scans. The study sample composed of 20 individuals (10 males and 10 females) for whom 120 scans featuring grimaces associated with disgust, surprise, "u" sound, smile and wide smile were collected by an optical scanner Vectra XT. In order to quantify the dissimilarity among 3D images, surface comparison approach based on aligned 3D meshes and closest point-to-point distances was carried out in Fidentis Analyst application. The study revealed that sexual dimorphism was indeed one of the factors which determined the extent and characteristics of facial deformations recorded for the studied expressions. In order to produce a grimace, males showed a tendency towards extending their facial movements while females were generally more restrained. Furthermore, the facial movements linked to the wide smile and "u" sound were revealed as the most extensive relative to the other expressions, while the smile and surprise were shown indistinguishable from the neutral face.

Depth accuracy of the RealSense F200: Low-cost 4D facial imaging

The RealSense F200 represents a new generation of economically viable 4-dimensional imaging (4D) systems for home use. However, its 3D geometric (depth) accuracy has not been clinically tested. Therefore, this study determined the depth accuracy of the RealSense, in a cohort of patients with a unilateral facial palsy (n = 34), by using the clinically validated 3dMD system as a gold standard. The patients were simultaneously recorded with both systems, capturing six Sunnybrook poses. This study has shown that the RealSense depth accuracy was not affected by a facial palsy (1.48 ± 0.28 mm), compared to a healthy face (1.46 ± 0.26 mm). Furthermore, the Sunnybrook poses did not influence the RealSense depth accuracy (p = 0.76). However, the distance of the patients to the RealSense was shown to affect the accuracy of the system, where the highest depth accuracy of 1.07 mm was measured at a distance of 35 cm. Overall, this study has shown that the RealSense can provide reliable and accurate depth data when recording a range of facial movements. Therefore, when the portability, low-costs, and availability of the RealSense are taken into consideration, the camera is a viable option for 4D close range imaging in telehealth.

Dynamic facial asymmetry in patients with repaired cleft lip using 4D imaging (video stereophotogrammetry)

BACKGROUND

Unilateral cleft lip is a profoundly asymmetrical condition affecting all hard and soft tissue layers from the nose to the upper lip. Although the asymmetry is minimized through cleft lip repair, a degree of asymmetry inevitably persists. Studies investigating asymmetry in patients with cleft lip have used facial measurements, and static 2D and 3D photography. The nose/lip/mouth area, however, is rarely static in our day to day social interactions.

METHODS

Non-syndromic patients with cleft lip and palate, and a control group without orofacial clefts underwent 50 frames per second 4D imaging while generating facial expressions including smiling and pouting. Key landmarks were tracked throughout the expression, corrected for head movement and a motion path of each landmark was generated. Asymmetry was assessed for both extent of displacement, using Euclidean distances between frames, and the shape of the motion path using Procrustes analysis.

RESULTS

Twelve patients were compared in each group with an age range from 8 to 18. Comparing the motion path of key landmarks in the upper lip demonstrated statistically significant differences in both the magnitude and shape of motion during smiling and pouting between cleft and non-cleft groups.

CONCLUSION

Video stereophotogrammetry of the repaired cleft lip demonstrates asymmetry of both the magnitude of motion as well as asymmetry of the path of the motion itself. This may be due to the effect of the scar tissue from the repair, from the abnormal anatomy involved with cleft lip or a combination of the two.

Breast volumetric analysis for aesthetic planning in breast reconstruction: a literature review of techniques

BACKGROUND

Accurate volumetric analysis is an essential component of preoperative planning in both reconstructive and aesthetic breast procedures towards achieving symmetrization and patient-satisfactory outcome. Numerous comparative studies and reviews of individual techniques have been reported. However, a unifying review of all techniques comparing their accuracy, reliability, and practicality has been lacking.

METHODS

A review of the published English literature dating from 1950 to 2015 using databases, such as PubMed, Medline, Web of Science, and EMBASE, was undertaken.

RESULTS

Since Bouman's first description of water displacement method, a range of volumetric assessment techniques have been described: thermoplastic casting, direct anthropomorphic measurement, two-dimensional (2D) imaging, and computed tomography (CT)/magnetic resonance imaging (MRI) scans. However, most have been unreliable, difficult to execute and demonstrate limited practicability. Introduction of 3D surface imaging has revolutionized the field due to its ease of use, fast speed, accuracy, and reliability. However, its widespread use has been limited by its high cost and lack of high level of evidence. Recent developments have unveiled the first web-based 3D surface imaging program, 4D imaging, and 3D printing.

CONCLUSIONS

Despite its importance, an accurate, reliable, and simple breast volumetric analysis tool has been elusive until the introduction of 3D surface imaging technology. However, its high cost has limited its wide usage. Novel adjunct technologies, such as web-based 3D surface imaging program, 4D imaging, and 3D printing, appear promising.

Evaluation of the Reproducibility of Nonverbal Facial Expressions Using a 3D Motion Capture System

Objective

To evaluate the reproducibility of three nonverbal facial expressions using a three-dimensional motion capture system.

Design

Prospective, cross-sectional, controlled study.

Setting

Glasgow Dental Hospital and School, University of Glasgow, United Kingdom.

Patients and Participants

Thirty-two subjects, 16 males and 16 females.

Methods

With a three-dimensional video passive stereophotogrammetry imaging system, maximal smile, cheek puff, and lip purse were captured for each subject. Anatomical facial landmarks were digitized on the first frame and then tracked automatically. The same facial expressions were captured 15 minutes later.

Main Outcome Measures

The magnitude of each expression and speed of landmark displacement were calculated. The landmark motion curves were spatially and temporally aligned to calculate the similarity of the dynamic movements of the same landmarks between the captures.

Results

There were no significant differences between individuals for magnitude ( P = .892) or for speed ( P = .456). There were significant differences in landmark movement similarity ( P = .011); similarity was more reproducible for maximal smile. There was no significant gender effect on the difference in magnitude. There was a significant gender effect on speed to reach maximal smile ( P = .044) and a pursed-lip expression ( P = .038). There was a significant gender effect on landmark movement similarities ( P = .031) for cheek puff expression.

Conclusions

There were no differences in magnitude and speed for maximal smile, cheek puff, and lip purse between the two captures for all participants. For individual expressions, maximal smile expression had the highest similarity value for individual landmarks.

Quantitative anatomical analysis of facial expression using a 3D motion capture system: Application to cosmetic surgery and facial recognition technology

The topography of the facial muscles differs between males and females and among individuals of the same gender. To explain the unique expressions that people can make, it is important to define the shapes of the muscle, their associations with the skin, and their relative functions. Three-dimensional (3D) motion-capture analysis, often used to study facial expression, was used in this study to identify characteristic skin movements in males and females when they made six representative basic expressions. The movements of 44 reflective markers (RMs) positioned on anatomical landmarks were measured. Their mean displacement was large in males [ranging from 14.31 mm (fear) to 41.15 mm (anger)], and 3.35-4.76 mm smaller in females [ranging from 9.55 mm (fear) to 37.80 mm (anger)]. The percentages of RMs involved in the ten highest mean maximum displacement values in making at least one expression were 47.6% in males and 61.9% in females. The movements of the RMs were larger in males than females but were more limited. Expanding our understanding of facial expression requires morphological studies of facial muscles and studies of related complex functionality. Conducting these together with quantitative analyses, as in the present study, will yield data valuable for medicine, dentistry, and engineering, for example, for surgical operations on facial regions, software for predicting changes in facial features and expressions after corrective surgery, and the development of face-mimicking robots.

The clinical application of three-dimensional motion capture (4D): a novel approach to quantify the dynamics of facial animations

The aim of this pilot study was to evaluate the feasibility of measuring the change in magnitude, speed, and motion similarity of facial animations in head and neck oncology patients, before and after lip split mandibulotomy. Seven subjects (four males, three females) aged 42-80 years were recruited. The subjects were asked to perform four facial animations (maximal smile, lip purse, cheek puff, and grimace) from rest to maximal position. The animations were captured using a Di4D motion capture system, which recorded 60 frames/s. Nine facial soft tissue landmarks were manually digitized on the first frame of the three-dimensional image of each animation by the same operator and were tracked automatically for the sequential frames. The intra-operator digitization error was within 0.4mm. Lip purse and maximal smile animations showed the least amount of change in magnitude (0.2mm) following surgery; speed difference was least for smile animation (-0.1mm/s). Motion similarity was found to be highest for lip purse animation (0.78). This pilot study confirmed that surgery did influence the dynamics of facial animations, and the Di4D capture system can be regarded as a feasible objective tool for assessing the impact of surgical interventions on facial soft tissue movements.