Measuring visual attention to faces with cleft deformity

Correlation of Orthognathic Surgical Movements to Perception of Facial Appearance in Patients With Cleft Lip and Palate

STUDY DESIGN

Cephalometric scans were compared before and after surgery to assess the degree of correction. Correlations between skeletal movements and survey outcomes were determined using multivariate regression analysis.

OBJECTIVE

This study aims to identify relationships between subjective observer-reported improvements in esthetics and emotional appearance with specific surgical movements.

METHODS

Ten patients at a single tertiary institution (average age: 18.1 ± 0.8), 9 males and 1 female, underwent orthognathic repair and had comprehensive cephalometric records. Standardized anterior posterior and lateral pre and postoperative photographs of patients were included in a survey to clinicians to assess noncognitive domains on a Likert Scale (1-10). CLEFT-Q was administered to gauge patient satisfaction in categories of appearance, speech, and quality of life.

RESULTS

Per clinicians, multiple domains increased including facial attractiveness (4.1 ± 0.7 versus 7.3 ± 0.7, P < 0.001), friendliness (4.5 ± 0.4 versus 7.3 ± 0.5, P < 0.001), confidence (4.1 ± 0.4 versus 7.1 ± 0.4, P < 0.001), and recommendation for surgery decreased (8.9 ± 0.1 versus 3.6 ± 0.5, P < 0.001). Speech distress decreased with increased SNA and convexity, whereas Psychological and Social scores decreased with an increased ANB. Functional eating and drinking scores increased with maxillary depth.

CONCLUSIONS

Orthognathic surgery improves many noncognitive domains in patients with cleft lip and palate as assessed by both patients and clinicians on all aspects of facial attractiveness and perception. These findings demonstrate objective bases of skeletal adjustments for perceived improvements in facial appearance and emotion.

Perception of aesthetic features after surgical treatment of craniofacial malformations by observers of the same age: An eye-tracking study

The aim of this study is to evaluate where exactly children and adolescents of the same group look when they interact with each other, and attempt to record and analyse the data recorded by eye-tracking technology.

MATERIALS AND METHODS

60 subjects participated in the study, evenly divided into three age categories of 20 each in pre-school/primary school age (5-9 years), early adolescence (10-14 years) and late adolescence/transition to adulthood (15-19 years). Age groups were matched and categorized to be used both for creating the picture series and testing. Photographs of patients with both unilateral and bilateral cleft lip and palate were used to create the series of images which consisted of a total of 15 photos, 5 of which were photos of patients with surgically treated cleft deformity and 10 control photos with healthy faces, that were presented in random order. Using the eye-tracking module, the data on "area of first view" (area of initial attention), "area with longest view" (area of sustained attention), "time until view in this area" (time of initial attention) and "frequency of view in each area" (time of sustained attention) were calculated.

RESULTS

Across all groups, there was no significant difference for the individual regions for the parameters of initial attention (area of first view), while the time until first fixation of one of the AOIs (time until view in this area) was significant for all facial regions. A predictable path of the facial scan is abandoned when secondary facial deformities are present and attention is focused more on the region of an existing deformity, which are the nose and mouth regions.

CONCLUSIONS

There are significant differences in both male and female participants' viewing of faces with and without secondary cleft deformity. While in the age group of the younger test persons it was still the mouth region that received special attention from the male viewers, this shifted in the male test persons of the middle age group to the nose region, which was fixed significantly more often and faster. In the female participants, the mouth and nose regions were each looked at for twice as long compared to the healthy faces, making both the mouth and the nose region are in the focus of observation.

A Narrative Review of Eye-Tracking Assessment of Esthetic Endpoints in Plastic, Reconstructive, and Craniofacial Surgery

ABSTRACT

Reporting of esthetic outcomes in plastic surgery relies classically on ill-defined endpoints and subjective measures very often from the patients' and/or providers' perspectives that are notoriously characterized by subjectivity and questionable solid scientific validation. With the recent trend of increasing demand for all types of esthetic medical and surgical interventions, there is an urgent need for reliable and objective outcome measures to quantitate esthetic outcomes and determine the efficacy of these interventions. The current review is intended to analyze available evidence regarding advantages as well as limitations of eye-tracking (ET) technology in objectively documenting esthetic outcomes of plastic, reconstructive, and craniofacial interventions. Although gaze pattern analysis is gaining more attention, ET data should be interpreted with caution; how a specific visual stimulus directly influences one's sense of esthetics is still not clear. Furthermore, despite its great potentials, it is still too early to confirm or deny ET usefulness. Nevertheless, patient-reported outcomes being most indicative of an esthetic intervention success, measurement of patients' satisfaction by ET technology could offer a major breakthrough in objective assessment of esthetic outcomes that need further in-depth investigation.

EVIDENCE LEVEL

Level III.

Impact of Surgical Rejuvenation on Visual Processing and Character Attribution of Faces

Background

This study considers observers' reflexive responses to the rejuvenated face, and how instinctive responses relate to subjective judgment. We investigated observers' reflexive perception of faces both pre and post surgical intervention during the early stages of visual processing. Subjective character attribution for all test images was also assessed by the same observers.

Method

Forty frontal facial images of 20 patients portraying the pre- and postoperative high superficial musculoaponeurotic system facelift along with variable concomitant procedures were studied. Nineteen lookzone regions were mapped post hoc onto each image. Forty observers examined the images, whereas an eye-tracking camera recorded their eye movements. Visual fixation data were recorded and analyzed. Observers also rated each image on the basis of five elemental positive character attributes.

Results

A statistically coherent but nonsignificant (P > 0.05) trend was identified with the surgical intervention resulting in greater attention being paid to the central triangle region of the face with reduction in attention to the facial periphery. Facial rejuvenation significantly increased the subjective character ratings of all five positively valenced attributes tested. Average age estimate of the photos decreased significantly from 54 to 48.6 years (true average age of 57.4 years).

Conclusions

We provide data illustrating both reflexive and subjective responses to facial rejuvenation. Observers reported a more favorable impression of the treated faces and evaluated them as being younger than their true age. A trend was detected for increased visual fixation of the central facial region following rejuvenation. Interpretation of these findings and indication for further research is provided.

The Use of Eye-Tracking Technology in Pediatric Orofacial Clefts: A Systematic Review and Meta-Analysis

This systematic review and meta-analysis assessed the quality of the peer-reviewed literature and evaluated the usefulness of eye-tracking technology in evaluating observers' perceptions of pediatric patients with orofacial clefts. PubMed, Science Direct, Wiley, and Web of Science were searched. Articles were screened in accordance with the Preferred Reporting Items for Systematic Review and Meta-analysis guidelines, and their methodological quality was assessed. Of the 10,254 identified studies, 12 were included. Eleven studies were cross-sectional, and one was a prospective cohort study. The main areas of interest analyzed were the eyes, nose, and mouth. Nine studies used assessment scales to analyze the link between perceived attractiveness and visualization patterns and measures. For the fixation duration outcome, six studies were eligible for inclusion in the meta-analysis. All studies reported on fixation duration in milliseconds and reported on a standard deviation. The meta-analysis demonstrated a significant difference in the measurements between the control groups and the patients with orofacial clefts. This might indicate the usefulness of eye-tracking technology as a metric for assessing the success of cleft repairs based on the perceptions of different populations. Future studies should be comprehensively reported on for comparability and reproducibility purposes.

Unsupervised anomaly appraisal of cleft faces using a StyleGAN2-based model adaptation technique

A novel machine learning framework that is able to consistently detect, localize, and measure the severity of human congenital cleft lip anomalies is introduced. The ultimate goal is to fill an important clinical void: to provide an objective and clinically feasible method of gauging baseline facial deformity and the change obtained through reconstructive surgical intervention. The proposed method first employs the StyleGAN2 generative adversarial network with model adaptation to produce a normalized transformation of 125 faces, and then uses a pixel-wise subtraction approach to assess the difference between all baseline images and their normalized counterparts (a proxy for severity of deformity). The pipeline of the proposed framework consists of the following steps: image preprocessing, face normalization, color transformation, heat-map generation, morphological erosion, and abnormality scoring. Heatmaps that finely discern anatomic anomalies visually corroborate the generated scores. The proposed framework is validated through computer simulations as well as by comparison of machine-generated versus human ratings of facial images. The anomaly scores yielded by the proposed computer model correlate closely with human ratings, with a calculated Pearson's r score of 0.89. The proposed pixel-wise measurement technique is shown to more closely mirror human ratings of cleft faces than two other existing, state-of-the-art image quality metrics (Learned Perceptual Image Patch Similarity and Structural Similarity Index). The proposed model may represent a new standard for objective, automated, and real-time clinical measurement of faces affected by congenital cleft deformity.

The Use of Eye-tracking Technology in Cleft Lip: A Literature Review

Eye-tracking has become an increasingly popular research tool within the field of cleft lip and/or palate (CL+/-P). Despite this, there are no standardized protocols for conducting research. Our objective was to conduct a literature review of the methodology and outcomes of previous publications using eye-tracking in CL+/-P.

Methods

The PubMed, Google Scholar, and Cochrane databases were searched to identify all articles published up to August 2022. All articles were screened by two independent reviewers. Inclusion criteria included using eye-tracking, image stimuli of CL+/-P, and outcome reporting using areas of interest (AOIs). Exclusion criteria included non-English studies, conference articles, and image stimuli of conditions other than CL+/-P.

Results

Forty articles were identified, and 16 met the inclusion/exclusion criteria. Thirteen studies only displayed images of individuals following cleft lip surgery with three only displaying unrepaired cleft lips. Significant variation was found in study design, particularly in the AOIs used to report gaze outcomes. Ten studies asked participants to provide an outcome score alongside eye-tracking; however, only four compared outcome data to eye-tracking data. This review is primarily limited by the minimal number of publications in this area.

Conclusions

Eye-tracking can be a powerful tool in evaluating appearance outcomes following CL+/-P surgery. It is currently limited by the lack of standardized research methodology and varied study design. Before future work, a replicable protocol should be developed to maximize the potential of this technology.

Parents of Children with Cleft Lip Exhibit Heightened Visual Attention to the Perioral Area

Following high-quality surgical repair, children born with a cleft lip anomaly may still display lasting visual differences. We exposed control adults and parents of affected children to images of children with cleft deformity and compared their visual tracking patterns. The protocol investigated whether parental exposure to secondary cleft deformity heightens or diminishes visual attraction to this type of structural facial variation.

Method

Twenty participants (10 control adults, 10 parents of affected children) assessed 40 colored images of children's faces while their eye movements were tracked. Twenty-four control images and 16 repaired cleft lip images were displayed to observers. Nine bilateral facial aesthetic zones were considered as regions of interest. Percentage of time visually fixating within each region, and statistical differences in fixation duration percentage between the two participant groups and across the bilateral regions of interest were analyzed.

Results

While both groups of observers directed more visual attention to the nasal and oral regions of the cleft images than control images, parents of children with cleft lip spent significantly more time fixating on these areas (25% and 24% of the time, respectively) than did unaffected adults (14.6% and 19.3%; P < 0.001).

Conclusions

These results demonstrate that parents of cleft lip children exhibit heightened attention to this type of facial difference relative to the naive observer. These findings highlight that observer profile can meaningfully influence the perception of a facial deformity. Awareness of this information may enhance communication between surgeon and parents of an affected child by providing added insight into parental perspective.

Toward a Universal Measure of Facial Difference Using Two Novel Machine Learning Models

A sensitive, objective, and universally accepted method of measuring facial deformity does not currently exist. Two distinct machine learning methods are described here that produce numerical scores reflecting the level of deformity of a wide variety of facial conditions.

METHODS

The first proposed technique utilizes an object detector based on a cascade function of Haar features. The model was trained using a dataset of 200,000 normal faces, as well as a collection of images devoid of faces. With the model trained to detect normal faces, the face detector confidence score was shown to function as a reliable gauge of facial abnormality. The second technique developed is based on a deep learning architecture of a convolutional autoencoder trained with the same rich dataset of normal faces. Because the convolutional autoencoder regenerates images disposed toward their training dataset (ie, normal faces), we utilized its reconstruction error as an indicator of facial abnormality. Scores generated by both methods were compared with human ratings obtained using a survey of 80 subjects evaluating 60 images depicting a range of facial deformities [rating from 1 (abnormal) to 7 (normal)].

RESULTS

The machine scores were highly correlated to the average human score, with overall Pearson's correlation coefficient exceeding 0.96 (P < 0.00001). Both methods were computationally efficient, reporting results within 3 seconds.

CONCLUSIONS

These models show promise for adaptation into a clinically accessible handheld tool. It is anticipated that ongoing development of this technology will facilitate multicenter collaboration and comparison of outcomes between conditions, techniques, operators, and institutions.

Personalized quantification of facial normality: a machine learning approach

What is a normal face? A fundamental task for the facial reconstructive surgeon is to answer that question as it pertains to any given individual. Accordingly, it would be important to be able to place the facial appearance of a patient with congenital or acquired deformity numerically along their own continuum of normality, and to measure any surgical changes against such a personalized benchmark. This has not previously been possible. We have solved this problem by designing a computerized model that produces realistic, normalized versions of any given facial image, and objectively measures the perceptual distance between the raw and normalized facial image pair. The model is able to faithfully predict human scoring of facial normality. We believe this work represents a paradigm shift in the assessment of the human face, holding great promise for development as an objective tool for surgical planning, patient education, and as a means for clinical outcome measurement.

Eye-Tracking Technology in Plastic and Reconstructive Surgery: A Systematic Review

BACKGROUND

The use of eye-tracking technology in plastic surgery has gained popularity over the past decade due to its ability to assess observers' visual preferences in an objective manner.

OBJECTIVES

The goal of this study was to provide a comprehensive review of eye-tracking studies in plastic and reconstructive surgery, which can aid in the design and conduct of high-quality eye-tracking studies.

METHODS

Through application of Preferred Reporting Items for Systematic review and Meta-Analysis (PRISMA) guidelines, a comprehensive search of articles published on eye-tracking across several databases was conducted from January 1946 to January 2019. Inclusion criteria included studies evaluating the use of eye-tracking technology in the field of plastic and reconstructive surgery. The resulting publications were screened by 2 independent reviewers.

RESULTS

A total of 595 articles were identified, 23 of which met our inclusion criteria. The most common application of eye-tracking was to assess individuals with cleft lip/palate (9 studies). All 19 studies that evaluated fixation patterns among conditions vs controls reported significant differences between the 2 groups. Five out of 7 studies assessing visual data between preoperative and postoperative patients identified significant differences between the preoperative and postoperative groups, whereas 2 studies did not. Nine studies examined the relation between severity indices, attractiveness scores, or personality ratings and gaze patterns. Correlation was found in 7 out of the 9 studies.

CONCLUSIONS

This systematic review demonstrates the utility of eye-tracking technology as a quantifiable objective assessment and emerging research tool for evaluating outcomes in several domains of plastic and reconstructive surgery.

Dermal Fat Grafting to Reconstruct the Parotidectomy Defect Normalizes Facial Attention

OBJECTIVES/HYPOTHESIS

Use validated eye-tracking technology to objectively measure 1) the attentional distraction of facial contour defects after superficial and total parotidectomy and 2) changes in attentional distraction with abdominal dermal fat graft reconstruction.

METHODS

Standardized frontal and oblique facial images of 16 patients who had undergone superficial or total parotidectomy with or without fat graft reconstruction; four normal controls were obtained. One hundred casual observers were recruited to view these images, and gaze data were collected using a Tobii Pro eye-tracking system. Gaze durations for predefined facial areas of interest were analyzed using mixed-effects linear regression to test study hypotheses.

RESULTS

For frontal images, total parotidectomy increased gaze to the operated parotid area compared to the contralateral nonoperated parotid area (92 milliseconds, 95% confidence interval [CI]: 48-138 milliseconds, P < .001). Fat grafting normalized the attentional distraction, with no difference in gaze time on the operated parotid region compared to normal control faces (P = .414). For oblique images, total parotidectomy increased gaze to the operated parotid area compared to the contralateral nonoperated parotid area (658 milliseconds, 95% CI: 463-854 milliseconds, P < .001). Fat grafting normalized this attentional distraction, with no difference in gaze time on the operated parotid region compared to normal control faces (P = .504). In both views, superficial parotidectomy demonstrated no significant attentional distractions, with or without fat grafting.

CONCLUSIONS

This eye-tracking study objectively demonstrates that total parotidectomy results in a facial contour deformity that is distracting to observers, whereas superficial parotidectomy does not. For total parotidectomy, this attentional distraction can be normalized with dermal fat graft reconstruction.

LEVEL OF EVIDENCE

3b Laryngoscope, 131:E124-E131, 2021.

The Use of Eye Tracking to Discern the Threshold at Which Metopic Orbitofrontal Deformity Attracts Attention

INTRODUCTION AND OBJECTIVES

Surgical treatment for trigonocephaly aims to eliminate a stigmatizing deformity, yet the severity that captures unwanted attention is unknown. Surgeons intervene at different points of severity, eliciting controversy. This study used eye tracking to investigate when deformity is perceived.

MATERIAL AND METHODS

Three-dimensional photogrammetric images of a normal child and a child with trigonocephaly were mathematically deformed, in 10% increments, to create a spectrum of 11 images. These images were shown to participants using an eye tracker. Participants' gaze patterns were analyzed, and participants were asked if each image looked "normal" or "abnormal."

RESULTS

Sixty-six graduate students were recruited. Average dwell time toward pathologic areas of interest (AOIs) increased proportionally, from 0.77 ± 0.33 seconds at 0% deformity to 1.08 ± 0.75 seconds at 100% deformity (P < .0001). A majority of participants did not agree an image looked "abnormal" until 90% deformity from any angle.

CONCLUSION

Eye tracking can be used as a proxy for attention threshold toward orbitofrontal deformity. The amount of attention toward orbitofrontal AOIs increased proportionally with severity. Participants did not generally agree there was "abnormality" until deformity was severe. This study supports the assertion that surgical intervention may be best reserved for more severe deformity.

Measuring Outcomes of Mohs Defect Reconstruction Using Eye-Tracking Technology

Importance

Objectively measuring how Mohs defect reconstruction changes casual observer attention has important implications for patients and facial plastic surgeons.

Objective

To use eye-tracking technology to objectively measure the ability of Mohs facial defect reconstruction to normalize facial attention.

Design, Setting, and Participants

This observational outcomes study was conducted at an academic tertiary referral center from January to June 2016. An eye-tracking system was used to record how 82 casual observers directed attention to photographs of 32 patients with Mohs facial defects of varying sizes and locations before and after reconstruction as well as 16 control faces with no facial defects. Statistical analysis was performed from November 2018 to January 2019.

Main Outcomes and Measures

First, the attentional distraction caused by facial defects was quantified in milliseconds of gaze time using eye tracking. Second, the eye-tracking data were analyzed using mixed-effects linear regression to assess the association of facial defect reconstruction with normalized facial attention.

Results

The 82 casual observers (63 women and 19 men; mean [SD] age, 34 [12] years) viewed control faces in a similar and consistent fashion, with most attention (65%; 95% CI, 62%-69%) directed at the central triangle, which includes the eyes, nose, and mouth. The eyes were the most visually important feature, capturing a mean of 60% (95% CI, 57%-64%) of fixation time within the central triangle and 39% (95% CI, 36%-43%) of total observer attention. The presence of Mohs defects was associated with statistically significant alterations in this pattern of normal facial attention. The larger the defect and the more centrally a defect was located, the more attentional distraction was observed, as measured by increased attention on the defect and decreased attention on the eyes, ranging from 729 (95% CI, 526-931) milliseconds for small peripheral defects to 3693 (95% CI, 3490-3896) milliseconds for large central defects. Reconstructive surgery was associated with improved gaze deviations for all faces and with normalized attention directed to the eyes for all faces except for those with large central defects.

Conclusions and Relevance

Mohs defects are associated with altered facial perception, diverting attention from valuable features such as the eyes. Reconstructive surgery was associated with normalized attentional distraction for many patients with cutaneous Mohs defects. These data are important to patients who want to know how reconstructive surgery could change the way people look at their face. The data also point to the possibility of outcomes prediction based on facial defect size and location before reconstruction. Eye tracking is a valuable research tool for outcomes assessment that lays the foundation for understanding how reconstructive surgery may change perception and normalize facial deformity.

Developing an Eye-Tracking Protocol to Determine Children’s Visual Perception of Secondary Cleft Lip Deformity

Objective:

To develop a protocol that will be used to measure children’s perception of secondary cleft lip deformity (SCLD) using objective eye-tracking technology.

Design:

Cross-sectional study. Data collection May and June of 2018.

Setting:

Single tertiary care pediatric hospital with a well-established cleft team.

Participants:

Participants were recruited from a general pediatric otolaryngology clinic. Sixty participants from 4 age groups (5-6, 10, 13, and 16 years) were enrolled on a voluntary basis.

Intervention:

Pediatric participants viewed images of children’s faces while wearing eye-tracking glasses. Ten images with unilateral SCLD and 2 control images with no facial scarring were viewed as gaze was assessed.

Main Outcome and Measure:

Successful gaze fixation was recorded across all age groups.

Results:

This article illustrates the types of data generated from glasses-based eye tracking in children. All children, regardless of age, spent more time with their gaze on a SCLD images (mean = 4.23 seconds; standard deviation [SD] = 1.41 seconds) compared to control images (mean = 3.97 seconds; SD = 1.42). Younger age groups spent less time looking at specific areas of interest in SCLD images.

Conclusion:

In this pilot study, we were able to successfully use eye-tracking technology in children to demonstrate gaze preference and a trend toward visual perception of SCLD changing with age. This protocol will allow for a future study, with larger and more diverse populations. Better understanding of how SCLD is perceived among children and adolescents has the potential to guide future interventions for SCLD and other facial deformities in pediatric patients.