Recognition of facial expression of fetuses by artificial intelligence (AI)

Bridging gaps in artificial intelligence adoption for maternal-fetal and obstetric care: Unveiling transformative capabilities and challenges

PURPOSE

This review aims to comprehensively explore the application of Artificial Intelligence (AI) to an area that has not been traditionally explored in depth: the continuum of maternal-fetal health. In doing so, the intent was to examine this physiologically continuous spectrum of mother and child health, as well as to highlight potential pitfalls, and suggest solutions for the same.

METHOD

A systematic search identified studies employing AI techniques for prediction, diagnosis, and decision support employing various modalities like imaging, electrophysiological signals and electronic health records in the domain of obstetrics and fetal health. In the selected articles then, AI applications in fetal morphology, gestational age assessment, congenital defect detection, fetal monitoring, placental analysis, and maternal physiological monitoring were critically examined both from the perspective of the domain and artificial intelligence.

RESULT

AI-driven solutions demonstrate promising capabilities in medical diagnostics and risk prediction, offering automation, improved accuracy, and the potential for personalized medicine. However, challenges regarding data availability, algorithmic transparency, and ethical considerations must be overcome to ensure responsible and effective clinical implementation. These challenges must be urgently addressed to ensure a domain as critical to public health as obstetrics and fetal health, is able to fully benefit from the gigantic strides made in the field of artificial intelligence.

CONCLUSION

Open access to relevant datasets is crucial for equitable progress in this critical public health domain. Integrating responsible and explainable AI, while addressing ethical considerations, is essential to maximize the public health benefits of AI-driven solutions in maternal-fetal care.

Fetal Face: Enhancing 3D Ultrasound Imaging by Postprocessing With AI Applications: Myth, Reality, or Legal Concerns?

The use of artificial intelligence (AI) platforms is revolutionizing the performance in managing metadata and big data. Medicine is another field where AI is spreading. However, this technological advancement is not amenable to errors or fraudulent misconducts. International organization and recently the European Union have released principles and recommendations for an appropriate use of AI in healthcare. In prenatal ultrasound diagnosis, the use of AI in daily practice is having a revolutionary impact. Notwithstanding, the diagnostic enhancement should be regulated, and AI applications should be developed to guarantee correct imaging acquisition and further postprocessing.

Using facial reaction analysis and machine learning to objectively assess the taste of medicines in children

For orally administered drugs, palatability is key in ensuring patient acceptability and treatment compliance. Therefore, understanding children's taste sensitivity and preferences can support formulators in making paediatric medicines more acceptable. Presently, we explore if the application of computer-vision techniques to videos of children's reaction to gustatory taste strips can provide an objective assessment of palatability. Children aged 4 to 11 years old tasted four different flavoured strips: no taste, bitter, sweet, and sour. Data was collected at home, under the supervision of a guardian, with responses recorded using the Aparito Atom app and smartphone camera. Participants scored each strip on a 5-point hedonic scale. Facial landmarks were identified in the videos, and quantitative measures, such as changes around the eyes, nose, and mouth, were extracted to train models to classify strip taste and score. We received 197 videos and 256 self-reported scores from 64 participants. The hedonic scale elicited expected results: children like sweetness, dislike bitterness and have varying opinions for sourness. The findings revealed the complexity and variability of facial reactions and highlighted specific measures, such as eyebrow and mouth corner elevations, as significant indicators of palatability. This study capturing children's objective reactions to taste sensations holds promise in identifying palatable drug formulations and assessing patient acceptability of paediatric medicines. Moreover, collecting data in the home setting allows for natural behaviour, with minimal burden for participants.

Repeatability and reproducibility of artificial intelligence-acquired fetal brain measurements (SonoCNS) in the second and third trimesters of pregnancy

Artificial Intelligence (AI)-based algorithms are increasingly entering clinical practice, aiding in the assessment of fetal anatomy and biometry. One such tool for evaluating the fetal head and central nervous system structures is SonoCNS™, which delineates appropriate planes for measuring head circumference (HC), biparietal diameter (BPD), occipitofrontal diameter (OFD), transcerebellar diameter (TCD), width of the posterior horn of the lateral ventricle (Vp), and cisterna magna (CM) based on a 3D volume acquired at the level of the fetal head's thalamic plane. This study aimed to evaluate the intra- and interobserver variability of measurements obtained using this software. The study included 381 patients, 270 in their second trimester of pregnancy (70%) and 111 in the third trimester. Each patient underwent manual biometric measurements of the aforementioned structures and twice using the SonoCNS software. We calculated the intraobserver variability between the manual measurements and the average of the automated measurements, as well as the interobserver variability for automated measurements. We also compared the median examination time for manual and automated measurements. The interclass correlation coefficients (ICC) for interobserver and intraobserver variability for parameters BPD, HC, and OFD ranged from good to excellent reproducibility in the general population and subgroups (> 0.75). CM and Vp measurements, both in the general population and subgroups, fell into the category of moderate (0.5-0.75) and poor reproducibility (< 0.5). TCD measurements showed moderate (> 0.5) to good reproducibility (0.75-0.9), and OFD showed good and excellent reproducibility. The assessment of the biometry of fetal head structures using SonoCNS took an average of 63 s compared to 14 s for manual measurement (p < 0.001). The SonoCNS™ software is characterized by good to excellent reproducibility and repeatability in the measurement of fetal skull biometry (BPD, HC, and OFD), with poorer performance in measurements of intracranial structures (CM, Vp, TCD). Apart from biometric parameters, the software is useful in clinical practice for delineating appropriate planes from the acquired volume of the fetal head and shortening examination time.

Fetal face shape analysis from prenatal 3D ultrasound images

3D ultrasound imaging of fetal faces has been predominantly confined to qualitative assessment. Many genetic conditions evade diagnosis and identification could assist with parental counselling, pregnancy management and neonatal care planning. We describe a methodology to build a shape model of the third trimester fetal face from 3D ultrasound and show how it can objectively describe morphological features and gestational-age related changes of normal fetal faces. 135 fetal face 3D ultrasound volumes (117 appropriately grown, 18 growth-restricted) of 24-34 weeks gestation were included. A 3D surface model of each face was obtained using a semi-automatic segmentation workflow. Size normalisation and rescaling was performed using a growth model giving the average size at every gestation. The model demonstrated a similar growth rate to standard head circumference reference charts. A landmark-free morphometry model was estimated to characterize shape differences using non-linear deformations of an idealized template face. Advancing gestation is associated with widening/fullness of the cheeks, contraction of the chin and deepening of the eyes. Fetal growth restriction is associated with a smaller average facial size but no morphological differences. This model may eventually be used as a reference to assist in the prenatal diagnosis of congenital anomalies with characteristic facial dysmorphisms.

Evolving the Era of 5D Ultrasound? A Systematic Literature Review on the Applications for Artificial Intelligence Ultrasound Imaging in Obstetrics and Gynecology

Artificial intelligence (AI) has gained prominence in medical imaging, particularly in obstetrics and gynecology (OB/GYN), where ultrasound (US) is the preferred method. It is considered cost effective and easily accessible but is time consuming and hindered by the need for specialized training. To overcome these limitations, AI models have been proposed for automated plane acquisition, anatomical measurements, and pathology detection. This study aims to overview recent literature on AI applications in OB/GYN US imaging, highlighting their benefits and limitations. For the methodology, a systematic literature search was performed in the PubMed and Cochrane Library databases. Matching abstracts were screened based on the PICOS (Participants, Intervention or Exposure, Comparison, Outcome, Study type) scheme. Articles with full text copies were distributed to the sections of OB/GYN and their research topics. As a result, this review includes 189 articles published from 1994 to 2023. Among these, 148 focus on obstetrics and 41 on gynecology. AI-assisted US applications span fetal biometry, echocardiography, or neurosonography, as well as the identification of adnexal and breast masses, and assessment of the endometrium and pelvic floor. To conclude, the applications for AI-assisted US in OB/GYN are abundant, especially in the subspecialty of obstetrics. However, while most studies focus on common application fields such as fetal biometry, this review outlines emerging and still experimental fields to promote further research.

Fetal brain activity and the free energy principle

OBJECTIVES

To study whether the free energy principle can explain fetal brain activity and the existence of fetal consciousness via a chaotic dimension derived using artificial intelligence.

METHODS

In this observational study, we used a four-dimensional ultrasound technique obtained to collect images of fetal faces from pregnancies at 27-37 weeks of gestation, between February and December 2021. We developed an artificial intelligence classifier that recognizes fetal facial expressions, which are thought to relate to fetal brain activity. We then applied the classifier to video files of facial images to generate each expression category's probabilities. We calculated the chaotic dimensions from the probability lists, and we created and investigated the free energy principle's mathematical model that was assumed to be linked to the chaotic dimension. We used a Mann-Whitney test, linear regression test, and one-way analysis of variance for statistical analysis.

RESULTS

The chaotic dimension revealed that the fetus had dense and sparse states of brain activity, which fluctuated at a statistically significant level. The chaotic dimension and free energy were larger in the sparse state than in the dense state.

CONCLUSIONS

The fluctuating free energy suggests consciousness seemed to exist in the fetus after 27 weeks.

An Innovative Three-Stage Model for Prenatal Genetic Disorder Detection Based on Region-of-Interest in Fetal Ultrasound

A global survey has revealed that genetic syndromes affect approximately 8% of the population, but most genetic diagnoses are typically made after birth. Facial deformities are commonly associated with chromosomal disorders. Prenatal diagnosis through ultrasound imaging is vital for identifying abnormal fetal facial features. However, this approach faces challenges such as inconsistent diagnostic criteria and limited coverage. To address this gap, we have developed FGDS, a three-stage model that utilizes fetal ultrasound images to detect genetic disorders. Our model was trained on a dataset of 2554 images. Specifically, FGDS employs object detection technology to extract key regions and integrates disease information from each region through ensemble learning. Experimental results demonstrate that FGDS accurately recognizes the anatomical structure of the fetal face, achieving an average precision of 0.988 across all classes. In the internal test set, FGDS achieves a sensitivity of 0.753 and a specificity of 0.889. Moreover, in the external test set, FGDS outperforms mainstream deep learning models with a sensitivity of 0.768 and a specificity of 0.837. This study highlights the potential of our proposed three-stage ensemble learning model for screening fetal genetic disorders. It showcases the model's ability to enhance detection rates in clinical practice and alleviate the burden on medical professionals.

A Deep-Learning-Based Method Can Detect Both Common and Rare Genetic Disorders in Fetal Ultrasound

A global survey indicates that genetic syndromes affect approximately 8% of the population, but most genetic diagnoses can only be performed after babies are born. Abnormal facial characteristics have been identified in various genetic diseases; however, current facial identification technologies cannot be applied to prenatal diagnosis. We developed Pgds-ResNet, a fully automated prenatal screening algorithm based on deep neural networks, to detect high-risk fetuses affected by a variety of genetic diseases. In screening for Trisomy 21, Trisomy 18, Trisomy 13, and rare genetic diseases, Pgds-ResNet achieved sensitivities of 0.83, 0.92, 0.75, and 0.96, and specificities of 0.94, 0.93, 0.95, and 0.92, respectively. As shown in heatmaps, the abnormalities detected by Pgds-ResNet are consistent with clinical reports. In a comparative experiment, the performance of Pgds-ResNet is comparable to that of experienced sonographers. This fetal genetic screening technology offers an opportunity for early risk assessment and presents a non-invasive, affordable, and complementary method to identify high-risk fetuses affected by genetic diseases. Additionally, it has the capability to screen for certain rare genetic conditions, thereby enhancing the clinic's detection rate.

A review on deep-learning algorithms for fetal ultrasound-image analysis

Deep-learning (DL) algorithms are becoming the standard for processing ultrasound (US) fetal images. A number of survey papers in the field is today available, but most of them are focusing on a broader area of medical-image analysis or not covering all fetal US DL applications. This paper surveys the most recent work in the field, with a total of 153 research papers published after 2017. Papers are analyzed and commented from both the methodology and the application perspective. We categorized the papers into (i) fetal standard-plane detection, (ii) anatomical structure analysis and (iii) biometry parameter estimation. For each category, main limitations and open issues are presented. Summary tables are included to facilitate the comparison among the different approaches. In addition, emerging applications are also outlined. Publicly-available datasets and performance metrics commonly used to assess algorithm performance are summarized, too. This paper ends with a critical summary of the current state of the art on DL algorithms for fetal US image analysis and a discussion on current challenges that have to be tackled by researchers working in the field to translate the research methodology into actual clinical practice.

Artificial intelligence to understand fluctuation of fetal brain activity by recognizing facial expressions

OBJECTIVE

To examine whether artificial intelligence can achieve discoveries regarding fetal brain activity.

METHODS

In this observational study, the authors collected images of fetal faces using a four-dimensional ultrasound technique obtained from singleton pregnancies of outpatients in routine practice at 27 to 37 weeks of gestation between February 1 and December 31, 2021. The authors developed an artificial intelligence classifier to recognize seven facial expressions of fetuses, then applied it to video files of fetal facial images to generate the probabilities, as confidence scores, of each expression category. Discrete Fourier transform and chaotic analysis were used to investigate the scores. Mann-Whitney test, t test, variance test, and one-way analysis of variance were used for statistical analysis.

RESULTS

Facial expression changes were observed in cycles averaging 66 to 73 s. The power spectrum showed that mouthing and neutral expressions were the most prevalent. There was a difference between categories for the spectrum (p = 0.004). Two different states--dense and sparse--of confidence scores were discovered. The correlation dimension was 1.19 ± 0.22 and 1.33 ± 0.27 for dense and sparse, respectively (p = 0.047).

CONCLUSION

This method objectively and quantitatively demonstrated fetal brain activity and may provide insight into how the fetus spends its time in utero.

Towards deep observation: A systematic survey on artificial intelligence techniques to monitor fetus via Ultrasound Images

Several reviews have been conducted regarding artificial intelligence (AI) techniques to improve pregnancy outcomes. But they are not focusing on ultrasound images. This survey aims to explore how AI can assist with fetal growth monitoring via ultrasound image. We reported our findings using the guidelines for PRISMA. We conducted a comprehensive search of eight bibliographic databases. Out of 1269 studies 107 are included. We found that 2D ultrasound images were more popular (88) than 3D and 4D ultrasound images (19). Classification is the most used method (42), followed by segmentation (31), classification integrated with segmentation (16) and other miscellaneous methods such as object-detection, regression, and reinforcement learning (18). The most common areas that gained traction within the pregnancy domain were the fetus head (43), fetus body (31), fetus heart (13), fetus abdomen (10), and the fetus face (10). This survey will promote the development of improved AI models for fetal clinical applications.

•

Artificial intelligence studies to monitor fetal development via ultrasound images

•

Fetal issues categorized based on four categories — general, head, heart, face, abdomen

•

The most used AI techniques are classification, segmentation, object detection, and RL

•

The research and practical implications are included.

Recent Advances in Machine Learning Applied to Ultrasound Imaging

Machine learning (ML) methods are pervading an increasing number of fields of application because of their capacity to effectively solve a wide variety of challenging problems. The employment of ML techniques in ultrasound imaging applications started several years ago but the scientific interest in this issue has increased exponentially in the last few years. The present work reviews the most recent (2019 onwards) implementations of machine learning techniques for two of the most popular ultrasound imaging fields, medical diagnostics and non-destructive evaluation. The former, which covers the major part of the review, was analyzed by classifying studies according to the human organ investigated and the methodology (e.g., detection, segmentation, and/or classification) adopted, while for the latter, some solutions to the detection/classification of material defects or particular patterns are reported. Finally, the main merits of machine learning that emerged from the study analysis are summarized and discussed.

Artificial intelligence in obstetrics

This study reviews recent advances on the application of artificial intelligence for the early diagnosis of various maternal-fetal conditions such as preterm birth and abnormal fetal growth. It is found in this study that various machine learning methods have been successfully employed for different kinds of data capture with regard to early diagnosis of maternal-fetal conditions. With the more popular use of artificial intelligence, ethical issues should also be considered accordingly.