Neural Networks for Classification and Image Generation of Aging in Genetic Syndromes

GestaltMatcher Database - A global reference for facial phenotypic variability in rare human diseases

The most important factor that complicates the work of dysmorphologists is the significant phenotypic variability of the human face. Next-Generation Phenotyping (NGP) tools that assist clinicians with recognizing characteristic syndromic patterns are particularly challenged when confronted with patients from populations different from their training data. To that end, we systematically analyzed the impact of genetic ancestry on facial dysmorphism. For that purpose, we established the GestaltMatcher Database (GMDB) as a reference dataset for medical images of patients with rare genetic disorders from around the world. We collected 10,980 frontal facial images - more than a quarter previously unpublished - from 8,346 patients, representing 581 rare disorders. Although the predominant ancestry is still European (67%), data from underrepresented populations have been increased considerably via global collaborations (19% Asian and 7% African). This includes previously unpublished reports for more than 40% of the African patients. The NGP analysis on this diverse dataset revealed characteristic performance differences depending on the composition of training and test sets corresponding to genetic relatedness. For clinical use of NGP, incorporating non-European patients resulted in a profound enhancement of GestaltMatcher performance. The top-5 accuracy rate increased by +11.29%. Importantly, this improvement in delineating the correct disorder from a facial portrait was achieved without decreasing the performance on European patients. By design, GMDB complies with the FAIR principles by rendering the curated medical data findable, accessible, interoperable, and reusable. This means GMDB can also serve as data for training and benchmarking. In summary, our study on facial dysmorphism on a global sample revealed a considerable cross ancestral phenotypic variability confounding NGP that should be counteracted by international efforts for increasing data diversity. GMDB will serve as a vital reference database for clinicians and a transparent training set for advancing NGP technology.

Approximating facial expression effects on diagnostic accuracy via generative AI in medical genetics

Artificial intelligence (AI) is increasingly used in genomics research and practice, and generative AI has garnered significant recent attention. In clinical applications of generative AI, aspects of the underlying datasets can impact results, and confounders should be studied and mitigated. One example involves the facial expressions of people with genetic conditions. Stereotypically, Williams (WS) and Angelman (AS) syndromes are associated with a "happy" demeanor, including a smiling expression. Clinical geneticists may be more likely to identify these conditions in images of smiling individuals. To study the impact of facial expression, we analyzed publicly available facial images of approximately 3500 individuals with genetic conditions. Using a deep learning (DL) image classifier, we found that WS and AS images with non-smiling expressions had significantly lower prediction probabilities for the correct syndrome labels than those with smiling expressions. This was not seen for 22q11.2 deletion and Noonan syndromes, which are not associated with a smiling expression. To further explore the effect of facial expressions, we computationally altered the facial expressions for these images. We trained HyperStyle, a GAN-inversion technique compatible with StyleGAN2, to determine the vector representations of our images. Then, following the concept of InterfaceGAN, we edited these vectors to recreate the original images in a phenotypically accurate way but with a different facial expression. Through online surveys and an eye-tracking experiment, we examined how altered facial expressions affect the performance of human experts. We overall found that facial expression is associated with diagnostic accuracy variably in different genetic conditions.

Recognition of Genetic Conditions After Learning With Images Created Using Generative Artificial Intelligence

Importance

The lack of standardized genetics training in pediatrics residencies, along with a shortage of medical geneticists, necessitates innovative educational approaches.

Objective

To compare pediatric resident recognition of Kabuki syndrome (KS) and Noonan syndrome (NS) after 1 of 4 educational interventions, including generative artificial intelligence (AI) methods.

Design, Setting, and Participants

This comparative effectiveness study used generative AI to create images of children with KS and NS. From October 1, 2022, to February 28, 2023, US pediatric residents were provided images through a web-based survey to assess whether these images helped them recognize genetic conditions.

Interventions

Participants categorized 20 images after exposure to 1 of 4 educational interventions (text-only descriptions, real images, and 2 types of images created by generative AI).

Main Outcomes and Measures

Associations between educational interventions with accuracy and self-reported confidence.

Results

Of 2515 contacted pediatric residents, 106 and 102 completed the KS and NS surveys, respectively. For KS, the sensitivity of text description was 48.5% (128 of 264), which was not significantly different from random guessing (odds ratio [OR], 0.94; 95% CI, 0.69-1.29; P = .71). Sensitivity was thus compared for real images vs random guessing (60.3% [188 of 312]; OR, 1.52; 95% CI, 1.15-2.00; P = .003) and 2 types of generative AI images vs random guessing (57.0% [212 of 372]; OR, 1.32; 95% CI, 1.04-1.69; P = .02 and 59.6% [193 of 324]; OR, 1.47; 95% CI, 1.12-1.94; P = .006) (denominators differ according to survey responses). The sensitivity of the NS text-only description was 65.3% (196 of 300). Compared with text-only, the sensitivity of the real images was 74.3% (205 of 276; OR, 1.53; 95% CI, 1.08-2.18; P = .02), and the sensitivity of the 2 types of images created by generative AI was 68.0% (204 of 300; OR, 1.13; 95% CI, 0.77-1.66; P = .54) and 71.0% (247 of 328; OR, 1.30; 95% CI, 0.92-1.83; P = .14). For specificity, no intervention was statistically different from text only. After the interventions, the number of participants who reported being unsure about important diagnostic facial features decreased from 56 (52.8%) to 5 (7.6%) for KS (P < .001) and 25 (24.5%) to 4 (4.7%) for NS (P < .001). There was a significant association between confidence level and sensitivity for real and generated images.

Conclusions and Relevance

In this study, real and generated images helped participants recognize KS and NS; real images appeared most helpful. Generated images were noninferior to real images and could serve an adjunctive role, particularly for rare conditions.

Comparison of clinical geneticist and computer visual attention in assessing genetic conditions

Artificial intelligence (AI) for facial diagnostics is increasingly used in the genetics clinic to evaluate patients with potential genetic conditions. Current approaches focus on one type of AI called Deep Learning (DL). While DL- based facial diagnostic platforms have a high accuracy rate for many conditions, less is understood about how this technology assesses and classifies (categorizes) images, and how this compares to humans. To compare human and computer attention, we performed eye-tracking analyses of geneticist clinicians (n = 22) and non-clinicians (n = 22) who viewed images of people with 10 different genetic conditions, as well as images of unaffected individuals. We calculated the Intersection-over-Union (IoU) and Kullback-Leibler divergence (KL) to compare the visual attentions of the two participant groups, and then the clinician group against the saliency maps of our deep learning classifier. We found that human visual attention differs greatly from DL model's saliency results. Averaging over all the test images, IoU and KL metric for the successful (accurate) clinician visual attentions versus the saliency maps were 0.15 and 11.15, respectively. Individuals also tend to have a specific pattern of image inspection, and clinicians demonstrate different visual attention patterns than non-clinicians (IoU and KL of clinicians versus non-clinicians were 0.47 and 2.73, respectively). This study shows that humans (at different levels of expertise) and a computer vision model examine images differently. Understanding these differences can improve the design and use of AI tools, and lead to more meaningful interactions between clinicians and AI technologies.

Privacy, bias and the clinical use of facial recognition technology: A survey of genetics professionals

Facial recognition technology (FRT) has been adopted as a precision medicine tool. The medical genetics field highlights both the clinical potential and privacy risks of this technology, putting the discipline at the forefront of a new digital privacy debate. Investigating how geneticists perceive the privacy concerns surrounding FRT can help shape the evolution and regulation of the field, and provide lessons for medicine and research more broadly. Five hundred and sixty-two genetics clinicians and researchers were approached to fill out a survey, 105 responded, and 80% of these completed. The survey consisted of 48 questions covering demographics, relationship to new technologies, views on privacy, views on FRT, and views on regulation. Genetics professionals generally placed a high value on privacy, although specific views differed, were context-specific, and covaried with demographic factors. Most respondents (88%) agreed that privacy is a basic human right, but only 37% placed greater weight on it than other values such as freedom of speech. Most respondents (80%) supported FRT use in genetics, but not necessarily for broader clinical use. A sizeable percentage (39%) were unaware of FRT's lower accuracy rates in marginalized communities and of the mental health effects of privacy violations (62%), but most (76% and 75%, respectively) expressed concern when informed. Overall, women and those who self-identified as politically progressive were more concerned about the lower accuracy rates in marginalized groups (88% vs. 64% and 83% vs. 63%, respectively). Younger geneticists were more wary than older geneticists about using FRT in genetics (28% compared to 56% "strongly" supported such use). There was an overall preference for more regulation, but respondents had low confidence in governments' or technology companies' ability to accomplish this. Privacy views are nuanced and context-dependent. Support for privacy was high but not absolute, and clear deficits existed in awareness of crucial FRT-related discrimination potential and mental health impacts. Education and professional guidelines may help to evolve views and practices within the field.

Computational facial analysis for rare Mendelian disorders

With the advances in computer vision, computational facial analysis has become a powerful and effective tool for diagnosing rare disorders. This technology, also called next-generation phenotyping (NGP), has progressed significantly over the last decade. This review paper will introduce three key NGP approaches. In 2014, Ferry et al. first presented Clinical Face Phenotype Space (CFPS) trained on eight syndromes. After 5 years, Gurovich et al. proposed DeepGestalt, a deep convolutional neural network trained on more than 21,000 patient images with 216 disorders. It was considered a state-of-the-art disorder classification framework. In 2022, Hsieh et al. developed GestaltMatcher to support the ultra-rare and novel disorders not supported in DeepGestalt. It further enabled the analysis of facial similarity presented in a given cohort or multiple disorders. Moreover, this article will present the usage of NGP for variant prioritization and facial gestalt delineation. Although NGP approaches have proven their capability in assisting the diagnosis of many disorders, many limitations remain. This article will introduce two future directions to address two main limitations: enabling the global collaboration for a medical imaging database that fulfills the FAIR principles and synthesizing patient images to protect patient privacy. In the end, with more and more NGP approaches emerging, we envision that the NGP technology can assist clinicians and researchers in diagnosing patients and analyzing disorders in multiple directions in the near future.

Application of facial analysis Technology in Clinical Genetics: Considerations for diverse populations

Facial analysis technology in rare diseases has the potential to shorten the diagnostic odyssey by providing physicians with a valuable diagnostic tool. Given that most clinical genetic resources focus on populations of European descent, we compare craniofacial features in genetic syndromes across different populations and review how machine learning algorithms perform on diagnosing genetic syndromes in geographically and ethnically diverse populations. We also discuss the value of populations from ancestrally diverse backgrounds in the training set of machine learning algorithms. Finally, this review demonstrates that across diverse population groups, machine learning models have outstanding accuracy as supported by the area under the curve values greater than 0.9. Artificial intelligence is only in its infancy in the diagnosis of rare disease in diverse populations and will become more accurate as larger and more diverse training sets, including a wider spectrum of ages, particularly infants, are studied.

Generative Methods for Pediatric Genetics Education

Artificial intelligence (AI) is used in an increasing number of areas, with recent interest in generative AI, such as using ChatGPT to generate programming code or DALL-E to make illustrations. We describe the use of generative AI in medical education. Specifically, we sought to determine whether generative AI could help train pediatric residents to better recognize genetic conditions. From publicly available images of individuals with genetic conditions, we used generative AI methods to create new images, which were checked for accuracy with an external classifier. We selected two conditions for study, Kabuki (KS) and Noonan (NS) syndromes, which are clinically important conditions that pediatricians may encounter. In this study, pediatric residents completed 208 surveys, where they each classified 20 images following exposure to one of 4 possible educational interventions, including with and without generative AI methods. Overall, we find that generative images perform similarly but appear to be slightly less helpful than real images. Most participants reported that images were useful, although real images were felt to be more helpful. We conclude that generative AI images may serve as an adjunctive educational tool, particularly for less familiar conditions, such as KS.

Human and computer attention in assessing genetic conditions

Deep learning (DL) and other types of artificial intelligence (AI) are increasingly used in many biomedical areas, including genetics. One frequent use in medical genetics involves evaluating images of people with potential genetic conditions to help with diagnosis. A central question involves better understanding how AI classifiers assess images compared to humans. To explore this, we performed eye-tracking analyses of geneticist clinicians and non-clinicians. We compared results to DL-based saliency maps. We found that human visual attention when assessing images differs greatly from the parts of images weighted by the DL model. Further, individuals tend to have a specific pattern of image inspection, and clinicians demonstrate different visual attention patterns than non-clinicians.

Perspectives on the future of dysmorphology

The field of clinical genetics and genomics continues to evolve. In the past few decades, milestones like the initial sequencing of the human genome, dramatic changes in sequencing technologies, and the introduction of artificial intelligence, have upended the field and offered fascinating new insights. Though difficult to predict the precise paths the field will follow, rapid change may continue to be inevitable. Within genetics, the practice of dysmorphology, as defined by pioneering geneticist David W. Smith in the 1960s as "the study of, or general subject of abnormal development of tissue form" has also been affected by technological advances as well as more general trends in biomedicine. To address possibilities, potential, and perils regarding the future of dysmorphology, a group of clinical geneticists, representing different career stages, areas of focus, and geographic regions, have contributed to this piece by providing insights about how the practice of dysmorphology will develop over the next several decades.

Investigating Determinants and Evaluating Deep Learning Training Approaches for Visual Acuity in Foveal Hypoplasia

Purpose

To describe the relationships between foveal structure and visual function in a cohort of individuals with foveal hypoplasia (FH) and to estimate FH grade and visual acuity using a deep learning classifier.

Design

Retrospective cohort study and experimental study.

Participants

A total of 201 patients with FH were evaluated at the National Eye Institute from 2004 to 2018.

Methods

Structural components of foveal OCT scans and corresponding clinical data were analyzed to assess their contributions to visual acuity. To automate FH scoring and visual acuity correlations, we evaluated the following 3 inputs for training a neural network predictor: (1) OCT scans, (2) OCT scans and metadata, and (3) real OCT scans and fake OCT scans created from a generative adversarial network.

Main Outcome Measures

The relationships between visual acuity outcomes and determinants, such as foveal morphology, nystagmus, and refractive error.

Results

The mean subject age was 24.4 years (range, 1–73 years; standard deviation = 18.25 years) at the time of OCT imaging. The mean best-corrected visual acuity (n = 398 eyes) was equivalent to a logarithm of the minimal angle of resolution (LogMAR) value of 0.75 (Snellen 20/115). Spherical equivalent refractive error (SER) ranged from −20.25 diopters (D) to +13.63 D with a median of +0.50 D. The presence of nystagmus and a high-LogMAR value showed a statistically significant relationship (P < 0.0001). The participants whose SER values were farther from plano demonstrated higher LogMAR values (n = 382 eyes). The proportion of patients with nystagmus increased with a higher FH grade. Variability in SER with grade 4 (range, −20.25 D to +13.00 D) compared with grade 1 (range, −8.88 D to +8.50 D) was statistically significant (P < 0.0001). Our neural network predictors reliably estimated the FH grading and visual acuity (correlation to true value > 0.85 and > 0.70, respectively) for a test cohort of 37 individuals (98 OCT scans). Training the predictor on real OCT scans with metadata and fake OCT scans improved the accuracy over the model trained on real OCT scans alone.

Conclusions

Nystagmus and foveal anatomy impact visual outcomes in patients with FH, and computational algorithms reliably estimate FH grading and visual acuity.