Large language models to facilitate pregnancy prediction after in vitro fertilization

We evaluated the efficacy of large language models (LLMs), specifically, generative pre-trained transformer-4 (GPT-4), in predicting pregnancy following in vitro fertilization (IVF) treatment and compared its accuracy with results from an original published study. Our findings revealed that GPT-4 can autonomously develop and refine advanced machine learning models for pregnancy prediction with minimal human intervention. The prediction accuracy was 0.79, and the area under the receiver operating characteristic curve (AUROC) was 0.89, exceeding or being at least equivalent to the metrics reported in the original study, that is, 0.78 for accuracy and 0.87 for AUROC. The results suggest that LLMs can facilitate data processing, optimize machine learning models in predicting IVF success rates, and provide data interpretation methods. This capacity can help bridge the knowledge gap between data scientists and medical personnel to solve the most pressing clinical challenges. However, more experiments on diverse and larger datasets are needed to validate and promote broader applications of LLMs in assisted reproduction.

Visual interpretability of image-based classification models by generative latent space disentanglement applied to in vitro fertilization

The success of deep learning in identifying complex patterns exceeding human intuition comes at the cost of interpretability. Non-linear entanglement of image features makes deep learning a "black box" lacking human meaningful explanations for the models' decision. We present DISCOVER, a generative model designed to discover the underlying visual properties driving image-based classification models. DISCOVER learns disentangled latent representations, where each latent feature encodes a unique classification-driving visual property. This design enables "human-in-the-loop" interpretation by generating disentangled exaggerated counterfactual explanations. We apply DISCOVER to interpret classification of in vitro fertilization embryo morphology quality. We quantitatively and systematically confirm the interpretation of known embryo properties, discover properties without previous explicit measurements, and quantitatively determine and empirically verify the classification decision of specific embryo instances. We show that DISCOVER provides human-interpretable understanding of "black box" classification models, proposes hypotheses to decipher underlying biomedical mechanisms, and provides transparency for the classification of individual predictions.

A generalized AI system for human embryo selection covering the entire IVF cycle via multi-modal contrastive learning

In vitro fertilization (IVF) has revolutionized infertility treatment, benefiting millions of couples worldwide. However, current clinical practices for embryo selection rely heavily on visual inspection of morphology, which is highly variable and experience dependent. Here, we propose a comprehensive artificial intelligence (AI) system that can interpret embryo-developmental knowledge encoded in vast unlabeled multi-modal datasets and provide personalized embryo selection. This AI platform consists of a transformer-based network backbone named IVFormer and a self-supervised learning framework, VTCLR (visual-temporal contrastive learning of representations), for training multi-modal embryo representations pre-trained on large and unlabeled data. When evaluated on clinical scenarios covering the entire IVF cycle, our pre-trained AI model demonstrates accurate and reliable performance on euploidy ranking and live-birth occurrence prediction. For AI vs. physician for euploidy ranking, our model achieved superior performance across all score categories. The results demonstrate the potential of the AI system as a non-invasive, efficient, and cost-effective tool to improve embryo selection and IVF outcomes.

Artificial intelligence-powered assisted ranking of sibling embryos to increase first cycle pregnancy rate

RESEARCH QUESTION

Could EMBRYOLY, an artificial intelligence embryo evaluation tool, assist embryologists to increase first cycle pregnancy rate and reduce cycles to pregnancy for patients?

DESIGN

Data from 11,988 embryos were collected via EMBRYOLY from 2666 egg retrievals (2019-2022) across 11 centres in France, Spain and Morocco using three time-lapse systems (TLS). Data from two independent clinics were also examined. EMBRYOLY's transformer-based model was applied to transferred embryos to evaluate ranking performances against pregnancy and birth outcomes. It was applied to cohorts to rank sibling embryos (including non-transferred) according to their likelihood of clinical pregnancy and to compute the agreement with the embryologist's highest ranked embryo. Its effect on time to pregnancy and first cycle pregnancy rate was evaluated on cohorts with multiple single blastocyst transfers, assuming the embryologist would have considered EMBRYOLY's ranking on the embryos favoured for transfer.

RESULTS

EMBRYOLY's score correlated significantly with clinical pregnancies and live births for cleavage and blastocyst transfers. This held true for clinical pregnancies from blastocyst transfers in two independent clinics. In cases of multiple single embryo transfers, embryologists achieved a 19.8% first cycle pregnancy rate, which could have been improved to 44.1% with the adjunctive use of EMBRYOLY (McNemar's test: P < 0.001). This could have reduced cycles to clinical pregnancy from 2.01 to 1.66 (Wilcoxon test: P < 0.001).

CONCLUSIONS

EMBRYOLY's potential to enhance first cycle pregnancy rates when combined with embryologists' expertise is highlighted. It reduces the number of unsuccessful cycles for patients across TLS and IVF centres.

Leveraging federated learning for boosting data privacy and performance in IVF embryo selection

PURPOSE

To study the effectiveness of federated learning in in vitro fertilization on embryo evaluation tasks.

METHODS

This is a retrospective cohort analysis. Two datasets were used in this study. The ploidy status dataset consisted of 10,065 embryo records, 3760 treatments, and 2479 infertile couples from 5 hospitals. The clinical pregnancy dataset consisted of 4495 embryo records, 4495 treatments, and 3704 infertile couples from 4 hospitals. Federated learning and the gradient boosting decision tree algorithm were utilized for modeling.

RESULTS

On the ploidy status dataset, the areas under the receiver operating characteristic curves of our model trained with federated learning were 71.78%, 73.10%, 69.39%, 69.72%, and 73.46% for 5 hospitals respectively, showing an average increase of 2.5% compared to those of our model trained without federated learning. On the clinical pregnancy dataset, the areas under the receiver operating characteristic curves of our model trained with federated learning were 72.03%, 56.77%, 61.63%, and 58.58% for 4 hospitals respectively, showing an average increase of 3.08%.

CONCLUSIONS

Federated learning can improve data privacy and data security and meanwhile improve the performance of embryo selection tasks by leveraging data from multiple sources. This study demonstrates the effectiveness of federated learning in embryo evaluation, and the results show the promise for future application.

Generative artificial intelligence to produce high-fidelity blastocyst-stage embryo images

STUDY QUESTION

Can generative artificial intelligence (AI) models produce high-fidelity images of human blastocysts?

SUMMARY ANSWER

Generative AI models exhibit the capability to generate high-fidelity human blastocyst images, thereby providing substantial training datasets crucial for the development of robust AI models.

WHAT IS KNOWN ALREADY

The integration of AI into IVF procedures holds the potential to enhance objectivity and automate embryo selection for transfer. However, the effectiveness of AI is limited by data scarcity and ethical concerns related to patient data privacy. Generative adversarial networks (GAN) have emerged as a promising approach to alleviate data limitations by generating synthetic data that closely approximate real images.

STUDY DESIGN, SIZE, DURATION

Blastocyst images were included as training data from a public dataset of time-lapse microscopy (TLM) videos (n = 136). A style-based GAN was fine-tuned as the generative model.

PARTICIPANTS/MATERIALS, SETTING, METHODS

We curated a total of 972 blastocyst images as training data, where frames were captured within the time window of 110-120 h post-insemination at 1-h intervals from TLM videos. We configured the style-based GAN model with data augmentation (AUG) and pretrained weights (Pretrained-T: with translation equivariance; Pretrained-R: with translation and rotation equivariance) to compare their optimization on image synthesis. We then applied quantitative metrics including Fréchet Inception Distance (FID) and Kernel Inception Distance (KID) to assess the quality and fidelity of the generated images. Subsequently, we evaluated qualitative performance by measuring the intelligence behavior of the model through the visual Turing test. To this end, 60 individuals with diverse backgrounds and expertise in clinical embryology and IVF evaluated the quality of synthetic embryo images.

MAIN RESULTS AND THE ROLE OF CHANCE

During the training process, we observed consistent improvement of image quality that was measured by FID and KID scores. Pretrained and AUG + Pretrained initiated with remarkably lower FID and KID values compared to both Baseline and AUG + Baseline models. Following 5000 training iterations, the AUG + Pretrained-R model showed the highest performance of the evaluated five configurations with FID and KID scores of 15.2 and 0.004, respectively. Subsequently, we carried out the visual Turing test, such that IVF embryologists, IVF laboratory technicians, and non-experts evaluated the synthetic blastocyst-stage embryo images and obtained similar performance in specificity with marginal differences in accuracy and sensitivity.

LIMITATIONS, REASONS FOR CAUTION

In this study, we primarily focused the training data on blastocyst images as IVF embryos are primarily assessed in blastocyst stage. However, generation of an array of images in different preimplantation stages offers further insights into the development of preimplantation embryos and IVF success. In addition, we resized training images to a resolution of 256 × 256 pixels to moderate the computational costs of training the style-based GAN models. Further research is needed to involve a more extensive and diverse dataset from the formation of the zygote to the blastocyst stage, e.g. video generation, and the use of improved image resolution to facilitate the development of comprehensive AI algorithms and to produce higher-quality images.

WIDER IMPLICATIONS OF THE FINDINGS

Generative AI models hold promising potential in generating high-fidelity human blastocyst images, which allows the development of robust AI models as it can provide sufficient training datasets while safeguarding patient data privacy. Additionally, this may help to produce sufficient embryo imaging training data with different (rare) abnormal features, such as embryonic arrest, tripolar cell division to avoid class imbalances and reach to even datasets. Thus, generative models may offer a compelling opportunity to transform embryo selection procedures and substantially enhance IVF outcomes.

STUDY FUNDING/COMPETING INTEREST(S)

This study was supported by a Horizon 2020 innovation grant (ERIN, grant no. EU952516) and a Horizon Europe grant (NESTOR, grant no. 101120075) of the European Commission to A.S. and M.Z.E., the Estonian Research Council (grant no. PRG1076) to A.S., and the EVA (Erfelijkheid Voortplanting & Aanleg) specialty program (grant no. KP111513) of Maastricht University Medical Centre (MUMC+) to M.Z.E.

TRIAL REGISTRATION NUMBER

Not applicable.

An artificial intelligence algorithm to select most viable embryos considering current process in IVF labs

Background

The most common Assisted Reproductive Technology is In-Vitro Fertilization (IVF). During IVF, embryologists commonly perform a morphological assessment to evaluate embryo quality and choose the best embryo for transferring to the uterus. However, embryo selection through morphological assessment is subjective, so various embryologists obtain different conclusions. Furthermore, humans can consider only a limited number of visual parameters resulting in a poor IVF success rate. Artificial intelligence (AI) for embryo selection is objective and can include many parameters, leading to better IVF outcomes.

Objectives

This study sought to use AI to (1) predict pregnancy results based on embryo images, (2) assess using more than one image of the embryo in the prediction of pregnancy but based on the current process in IVF labs, and (3) compare results of AI-Based methods and embryologist experts in predicting pregnancy.

Methods

A data set including 252 Time-lapse Videos of embryos related to IVF performed between 2017 and 2020 was collected. Frames related to 19 ± 1, 43 ± 1, and 67 ± 1 h post-insemination were extracted. Well-Known CNN architectures with transfer learning have been applied to these images. The results have been compared with an algorithm that only uses the final image of embryos. Furthermore, the results have been compared with five experienced embryologists.

Results

To predict the pregnancy outcome, we applied five well-known CNN architectures (AlexNet, ResNet18, ResNet34, Inception V3, and DenseNet121). DeepEmbryo, using three images, predicts pregnancy better than the algorithm that only uses one final image. It also can predict pregnancy better than all embryologists. Different well-known architectures can successfully predict pregnancy chances with up to 75.0% accuracy using Transfer Learning.

Conclusion

We have developed DeepEmbryo, an AI-based tool that uses three static images to predict pregnancy. Additionally, DeepEmbryo uses images that can be obtained in the current IVF process in almost all IVF labs. AI-based tools have great potential for predicting pregnancy and can be used as a proper tool in the future.

Dev-ResNet: automated developmental event detection using deep learning

Delineating developmental events is central to experimental research using early life stages, permitting widespread identification of changes in event timing between species and environments. Yet, identifying developmental events is incredibly challenging, limiting the scale, reproducibility and throughput of using early life stages in experimental biology. We introduce Dev-ResNet, a small and efficient 3D convolutional neural network capable of detecting developmental events characterised by both spatial and temporal features, such as the onset of cardiac function and radula activity. We demonstrate the efficacy of Dev-ResNet using 10 diverse functional events throughout the embryonic development of the great pond snail, Lymnaea stagnalis. Dev-ResNet was highly effective in detecting the onset of all events, including the identification of thermally induced decoupling of event timings. Dev-ResNet has broad applicability given the ubiquity of bioimaging in developmental biology, and the transferability of deep learning, and so we provide comprehensive scripts and documentation for applying Dev-ResNet to different biological systems.

WISE: whole-scenario embryo identification using self-supervised learning encoder in IVF

PURPOSE

To study the effectiveness of whole-scenario embryo identification using a self-supervised learning encoder (WISE) in in vitro fertilization (IVF) on time-lapse, cross-device, and cryo-thawed scenarios.

METHODS

WISE was based on the vision transformer (ViT) architecture and masked autoencoders (MAE), a self-supervised learning (SSL) method. To train WISE, we prepared three datasets including the SSL pre-training dataset, the time-lapse identification dataset, and the cross-device identification dataset. To identify whether pairs of images were from the same embryos in different scenarios in the downstream identification tasks, embryo images including time-lapse and microscope images were first pre-processed through object detection, cropping, padding, and resizing, and then fed into WISE to get predictions.

RESULTS

WISE could accurately identify embryos in the three scenarios. The accuracy was 99.89% on the time-lapse identification dataset, and 83.55% on the cross-device identification dataset. Besides, we subdivided a cryo-thawed evaluation set from the cross-device test set to have a better estimation of how WISE performs in the real-world, and it reached an accuracy of 82.22%. There were approximately 10% improvements in cross-device and cryo-thawed identification tasks after the SSL method was applied. Besides, WISE demonstrated improvements in the accuracy of 9.5%, 12%, and 18% over embryologists in the three scenarios.

CONCLUSION

SSL methods can improve embryo identification accuracy even when dealing with cross-device and cryo-thawed paired images. The study is the first to apply SSL in embryo identification, and the results show the promise of WISE for future application in embryo witnessing.

Knowledge-embedded spatio-temporal analysis for euploidy embryos identification in couples with chromosomal rearrangements

BACKGROUND

The goal of the assisted reproductive treatment is to transfer one euploid blastocyst and to help infertile women giving birth one healthy neonate. Some algorithms have been used to assess the ploidy status of embryos derived from couples with normal chromosome, who subjected to preimplantation genetic testing for aneuploidy (PGT-A) treatment. However, it is currently unknown whether artificial intelligence model can be used to assess the euploidy status of blastocyst derived from populations with chromosomal rearrangement.

METHODS

From February 2020 to May 2021, we collected the whole raw time-lapse videos at multiple focal planes from in vitro cultured embryos, the clinical information of couples, and the comprehensive chromosome screening results of those blastocysts that had received PGT treatment. Initially, we developed a novel deep learning model called the Attentive Multi-Focus Selection Network (AMSNet) to analyze time-lapse videos in real time and predict blastocyst formation. Building upon AMSNet, we integrated additional clinically predictive variables and created a second deep learning model, the Attentive Multi-Focus Video and Clinical Information Fusion Network (AMCFNet), to assess the euploidy status of embryos. The efficacy of the AMCFNet was further tested in embryos with parental chromosomal rearrangements. The receiver operating characteristic curve (ROC) was used to evaluate the superiority of the model.

RESULTS

A total of 4112 embryos with complete time-lapse videos were enrolled for the blastocyst formation prediction task, and 1422 qualified blastocysts received PGT-A ( n = 589) or PGT for chromosomal structural rearrangement (PGT-SR, n = 833) were enrolled for the euploidy assessment task in this study. The AMSNet model using seven focal raw time-lapse videos has the best real-time accuracy. The real-time accuracy for AMSNet to predict blastocyst formation reached above 70% on the day 2 of embryo culture, and then increased to 80% on the day 4 of embryo culture. Combing with 4 clinical features of couples, the AUC of AMCFNet with 7 focal points increased to 0.729 in blastocysts derived from couples with chromosomal rearrangement.

CONCLUSION

Integrating seven focal raw time-lapse images of embryos and parental clinical information, AMCFNet model have the capability of assessing euploidy status in blastocysts derived from couples with chromosomal rearrangement.

Application of artificial intelligence in gametes and embryos selection

Gamete and embryo quality are critical to the success rate of Assisted Reproductive Technology (ART) cycles, but there remains a lack of methods to accurately measure the quality of sperm, oocytes and embryos. The ability of Artificial Intelligence (AI) technology to analyze large amounts of data, especially video and images, is particularly useful in gamete and embryo assessment and selection. The well-trained model has fast calculation speed and high accuracy, which can help embryologists to perform more objective gamete and embryo selection. Various artificial intelligence models have been developed for gamete and embryo assessment, some of which exhibit good performance. In this review, we summarize the latest applications of AI technology in semen analysis, as well as selection for sperm, oocyte and embryo, and discuss the existing problems and development directions of artificial intelligence in this field.

Embryo Selection for IVF using Machine Learning Techniques Based on Light Microscopic Images of Embryo and Additional Factors

The current process of embryo selection in In Vitro Fertilization (IVF) process is based on morphological criteria, e.g., Istanbul scoring system and manually evaluated by embryologists; consequently, the assessment can be subjective. In the case of multiple embryos that have the same morphological grading, there is no guidance on how embryos should be prioritized to be transferred. This work aims to develop a deep learning-based model to classify viable and non-viable embryos using light microscopic images of an embryo. Additional features according to Istanbul grading system and the patients' age is also included in the model. Various models are evaluated and the best model based on the fusion of embryo images and additional features provides accuracy, sensitivity, and area under curve (AUC) of 65%, 74.29% and 0.72, respectively. The distributions of the prediction score corresponding to each additional feature are analysed and compared with pregnant and non-pregnant ground truths. We have found that the additional factors, such as age, embryo development stage, the quality of inner cell mass (ICM), and trophectoderm (TE) have a positive impact and enhanced the model prediction of implantation potential.

A hybrid artificial intelligence model leverages multi-centric clinical data to improve fetal heart rate pregnancy prediction across time-lapse systems

STUDY QUESTION

Can artificial intelligence (AI) algorithms developed to assist embryologists in evaluating embryo morphokinetics be enriched with multi-centric clinical data to better predict clinical pregnancy outcome?

SUMMARY ANSWER

Training algorithms on multi-centric clinical data significantly increased AUC compared to algorithms that only analyzed the time-lapse system (TLS) videos.

WHAT IS KNOWN ALREADY

Several AI-based algorithms have been developed to predict pregnancy, most of them based only on analysis of the time-lapse recording of embryo development. It remains unclear, however, whether considering numerous clinical features can improve the predictive performances of time-lapse based embryo evaluation.

STUDY DESIGN, SIZE, DURATION

A dataset of 9986 embryos (95.60% known clinical pregnancy outcome, 32.47% frozen transfers) from 5226 patients from 14 European fertility centers (in two countries) recorded with three different TLS was used to train and validate the algorithms. A total of 31 clinical factors were collected. A separate test set (447 videos) was used to compare performances between embryologists and the algorithm.

PARTICIPANTS/MATERIALS, SETTING, METHODS

Clinical pregnancy (defined as a pregnancy leading to a fetal heartbeat) outcome was first predicted using a 3D convolutional neural network that analyzed videos of the embryonic development up to 2 or 3 days of development (33% of the database) or up to 5 or 6 days of development (67% of the database). The output video score was then fed as input alongside clinical features to a gradient boosting algorithm that generated a second score corresponding to the hybrid model. AUC was computed across 7-fold of the validation dataset for both models. These predictions were compared to those of 13 senior embryologists made on the test dataset.

MAIN RESULTS AND THE ROLE OF CHANCE

The average AUC of the hybrid model across all 7-fold was significantly higher than that of the video model (0.727 versus 0.684, respectively, P = 0.015; Wilcoxon test). A SHapley Additive exPlanations (SHAP) analysis of the hybrid model showed that the six first most important features to predict pregnancy were morphokinetics of the embryo (video score), oocyte age, total gonadotrophin dose intake, number of embryos generated, number of oocytes retrieved, and endometrium thickness. The hybrid model was shown to be superior to embryologists with respect to different metrics, including the balanced accuracy (P ≤ 0.003; Wilcoxon test). The likelihood of pregnancy was linearly linked to the hybrid score, with increasing odds ratio (maximum P-value = 0.001), demonstrating the ranking capacity of the model. Training individual hybrid models did not improve predictive performance. A clinic hold-out experiment was conducted and resulted in AUCs ranging between 0.63 and 0.73. Performance of the hybrid model did not vary between TLS or between subgroups of embryos transferred at different days of embryonic development. The hybrid model did fare better for patients older than 35 years (P < 0.001; Mann-Whitney test), and for fresh transfers (P < 0.001; Mann-Whitney test).

LIMITATIONS, REASONS FOR CAUTION

Participant centers were located in two countries, thus limiting the generalization of our conclusion to wider subpopulations of patients. Not all clinical features were available for all embryos, thus limiting the performances of the hybrid model in some instances.

WIDER IMPLICATIONS OF THE FINDINGS

Our study suggests that considering clinical data improves pregnancy predictive performances and that there is no need to retrain algorithms at the clinic level unless they follow strikingly different practices. This study characterizes a versatile AI algorithm with similar performance on different time-lapse microscopes and on embryos transferred at different development stages. It can also help with patients of different ages and protocols used but with varying performances, presumably because the task of predicting fetal heartbeat becomes more or less hard depending on the clinical context. This AI model can be made widely available and can help embryologists in a wide range of clinical scenarios to standardize their practices.

STUDY FUNDING/COMPETING INTEREST(S)

Funding for the study was provided by ImVitro with grant funding received in part from BPIFrance (Bourse French Tech Emergence (DOS0106572/00), Paris Innovation Amorçage (DOS0132841/00), and Aide au Développement DeepTech (DOS0152872/00)). A.B.-C. is a co-owner of, and holds stocks in, ImVitro SAS. A.B.-C. and F.D.M. hold a patent for 'Devices and processes for machine learning prediction of in vitro fertilization' (EP20305914.2). A.D., N.D., M.M.F., and F.D.M. are or have been employees of ImVitro and have been granted stock options. X.P.-V. has been paid as a consultant to ImVitro and has been granted stocks options of ImVitro. L.C.-D. and C.G.-S. have undertaken paid consultancy for ImVitro SAS. The remaining authors have no conflicts to declare.

TRIAL REGISTRATION NUMBER

N/A.

A non-invasive artificial intelligence approach for the prediction of human blastocyst ploidy: a retrospective model development and validation study

BACKGROUND

One challenge in the field of in-vitro fertilisation is the selection of the most viable embryos for transfer. Morphological quality assessment and morphokinetic analysis both have the disadvantage of intra-observer and inter-observer variability. A third method, preimplantation genetic testing for aneuploidy (PGT-A), has limitations too, including its invasiveness and cost. We hypothesised that differences in aneuploid and euploid embryos that allow for model-based classification are reflected in morphology, morphokinetics, and associated clinical information.

METHODS

In this retrospective study, we used machine-learning and deep-learning approaches to develop STORK-A, a non-invasive and automated method of embryo evaluation that uses artificial intelligence to predict embryo ploidy status. Our method used a dataset of 10 378 embryos that consisted of static images captured at 110 h after intracytoplasmic sperm injection, morphokinetic parameters, blastocyst morphological assessments, maternal age, and ploidy status. Independent and external datasets, Weill Cornell Medicine EmbryoScope+ (WCM-ES+; Weill Cornell Medicine Center of Reproductive Medicine, NY, USA) and IVI Valencia (IVI Valencia, Health Research Institute la Fe, Valencia, Spain) were used to test the generalisability of STORK-A and were compared measuring accuracy and area under the receiver operating characteristic curve (AUC).

FINDINGS

Analysis and model development included the use of 10 378 embryos, all with PGT-A results, from 1385 patients (maternal age range 21-48 years; mean age 36·98 years [SD 4·62]). STORK-A predicted aneuploid versus euploid embryos with an accuracy of 69·3% (95% CI 66·9-71·5; AUC 0·761; positive predictive value [PPV] 76·1%; negative predictive value [NPV] 62·1%) when using images, maternal age, morphokinetics, and blastocyst score. A second classification task trained to predict complex aneuploidy versus euploidy and single aneuploidy produced an accuracy of 74·0% (95% CI 71·7-76·1; AUC 0·760; PPV 54·9%; NPV 87·6%) using an image, maternal age, morphokinetic parameters, and blastocyst grade. A third classification task trained to predict complex aneuploidy versus euploidy had an accuracy of 77·6% (95% CI 75·0-80·0; AUC 0·847; PPV 76·7%; NPV 78·0%). STORK-A reported accuracies of 63·4% (AUC 0·702) on the WCM-ES+ dataset and 65·7% (AUC 0·715) on the IVI Valencia dataset, when using an image, maternal age, and morphokinetic parameters, similar to the STORK-A test dataset accuracy of 67·8% (AUC 0·737), showing generalisability.

INTERPRETATION

As a proof of concept, STORK-A shows an ability to predict embryo ploidy in a non-invasive manner and shows future potential as a standardised supplementation to traditional methods of embryo selection and prioritisation for implantation or recommendation for PGT-A.

FUNDING

US National Institutes of Health.

Reporting on the Value of Artificial Intelligence in Predicting the Optimal Embryo for Transfer: A Systematic Review including Data Synthesis

Artificial intelligence (AI) has been gaining support in the field of in vitro fertilization (IVF). Despite the promising existing data, AI cannot yet claim gold-standard status, which serves as the rationale for this study. This systematic review and data synthesis aims to evaluate and report on the predictive capabilities of AI-based prediction models regarding IVF outcome. The study has been registered in PROSPERO (CRD42021242097). Following a systematic search of the literature in Pubmed/Medline, Embase, and Cochrane Central Library, 18 studies were identified as eligible for inclusion. Regarding live-birth, the Area Under the Curve (AUC) of the Summary Receiver Operating Characteristics (SROC) was 0.905, while the partial AUC (pAUC) was 0.755. The Observed: Expected ratio was 1.12 (95%CI: 0.26–2.37; 95%PI: 0.02–6.54). Regarding clinical pregnancy with fetal heartbeat, the AUC of the SROC was 0.722, while the pAUC was 0.774. The O:E ratio was 0.77 (95%CI: 0.54–1.05; 95%PI: 0.21–1.62). According to this data synthesis, the majority of the AI-based prediction models are successful in accurately predicting the IVF outcome regarding live birth, clinical pregnancy, clinical pregnancy with fetal heartbeat, and ploidy status. This review attempted to compare between AI and human prediction capabilities, and although studies do not allow for a meta-analysis, this systematic review indicates that the AI-based prediction models perform rather similarly to the embryologists’ evaluations. While AI models appear marginally more effective, they still have some way to go before they can claim to significantly surpass the clinical embryologists’ predictive competence.

Stain-free detection of embryo polarization using deep learning

Polarization of the mammalian embryo at the right developmental time is critical for its development to term and would be valuable in assessing the potential of human embryos. However, tracking polarization requires invasive fluorescence staining, impermissible in the in vitro fertilization clinic. Here, we report the use of artificial intelligence to detect polarization from unstained time-lapse movies of mouse embryos. We assembled a dataset of bright-field movie frames from 8-cell-stage embryos, side-by-side with corresponding images of fluorescent markers of cell polarization. We then used an ensemble learning model to detect whether any bright-field frame showed an embryo before or after onset of polarization. Our resulting model has an accuracy of 85% for detecting polarization, significantly outperforming human volunteers trained on the same data (61% accuracy). We discovered that our self-learning model focuses upon the angle between cells as one known cue for compaction, which precedes polarization, but it outperforms the use of this cue alone. By compressing three-dimensional time-lapsed image data into two-dimensions, we are able to reduce data to an easily manageable size for deep learning processing. In conclusion, we describe a method for detecting a key developmental feature of embryo development that avoids clinically impermissible fluorescence staining.

Deep Learning-Enabled Technologies for Bioimage Analysis

Deep learning (DL) is a subfield of machine learning (ML), which has recently demonstrated its potency to significantly improve the quantification and classification workflows in biomedical and clinical applications. Among the end applications profoundly benefitting from DL, cellular morphology quantification is one of the pioneers. Here, we first briefly explain fundamental concepts in DL and then we review some of the emerging DL-enabled applications in cell morphology quantification in the fields of embryology, point-of-care ovulation testing, as a predictive tool for fetal heart pregnancy, cancer diagnostics via classification of cancer histology images, autosomal polycystic kidney disease, and chronic kidney diseases.

Robust and generalizable embryo selection based on artificial intelligence and time-lapse image sequences

Assessing and selecting the most viable embryos for transfer is an essential part of in vitro fertilization (IVF). In recent years, several approaches have been made to improve and automate the procedure using artificial intelligence (AI) and deep learning. Based on images of embryos with known implantation data (KID), AI models have been trained to automatically score embryos related to their chance of achieving a successful implantation. However, as of now, only limited research has been conducted to evaluate how embryo selection models generalize to new clinics and how they perform in subgroup analyses across various conditions. In this paper, we investigate how a deep learning-based embryo selection model using only time-lapse image sequences performs across different patient ages and clinical conditions, and how it correlates with traditional morphokinetic parameters. The model was trained and evaluated based on a large dataset from 18 IVF centers consisting of 115,832 embryos, of which 14,644 embryos were transferred KID embryos. In an independent test set, the AI model sorted KID embryos with an area under the curve (AUC) of a receiver operating characteristic curve of 0.67 and all embryos with an AUC of 0.95. A clinic hold-out test showed that the model generalized to new clinics with an AUC range of 0.60-0.75 for KID embryos. Across different subgroups of age, insemination method, incubation time, and transfer protocol, the AUC ranged between 0.63 and 0.69. Furthermore, model predictions correlated positively with blastocyst grading and negatively with direct cleavages. The fully automated iDAScore v1.0 model was shown to perform at least as good as a state-of-the-art manual embryo selection model. Moreover, full automatization of embryo scoring implies fewer manual evaluations and eliminates biases due to inter- and intraobserver variation.

Predicting Embryo Viability Based on Self-Supervised Alignment of Time-Lapse Videos

With self-supervised learning, both labeled and unlabeled data can be used for representation learning and model pretraining. This is particularly relevant when automating the selection of a patient's fertilized eggs (embryos) during a fertility treatment, in which only the embryos that were transferred to the female uterus may have labels of pregnancy. In this paper, we apply a self-supervised video alignment method known as temporal cycle-consistency (TCC) on 38176 time-lapse videos of developing embryos, of which 14550 were labeled. We show how TCC can be used to extract temporal similarities between embryo videos and use these for predicting pregnancy likelihood. Our temporal similarity method outperforms the time alignment measurement (TAM) with an area under the receiver operating characteristic (AUC) of 0.64 vs. 0.56. Compared to existing embryo evaluation models, it places in between a pure temporal and a spatio-temporal model that both require manual annotations. Furthermore, we use TCC for transfer learning in a semi-supervised fashion and show significant performance improvements compared to standard supervised learning, when only a small subset of the dataset is labeled. Specifically, two variants of transfer learning both achieve an AUC of 0.66 compared to 0.63 for supervised learning when 16% of the dataset is labeled.

A New Few-Shot Learning Method of Bacterial Colony Counting Based on the Edge Computing Device

Bacterial colony counting is a time consuming but important task for many fields, such as food quality testing and pathogen detection, which own the high demand for accurate on-site testing. However, bacterial colonies are often overlapped, adherent with each other, and difficult to precisely process by traditional algorithms. The development of deep learning has brought new possibilities for bacterial colony counting, but deep learning networks usually require a large amount of training data and highly configured test equipment. The culture and annotation time of bacteria are costly, and professional deep learning workstations are too expensive and large to meet portable requirements. To solve these problems, we propose a lightweight improved YOLOv3 network based on the few-shot learning strategy, which is able to accomplish high detection accuracy with only five raw images and be deployed on a low-cost edge device. Compared with the traditional methods, our method improved the average accuracy from 64.3% to 97.4% and decreased the False Negative Rate from 32.1% to 1.5%. Our method could greatly improve the detection accuracy, realize the portability for on-site testing, and significantly save the cost of data collection and annotation over 80%, which brings more potential for bacterial colony counting.

Time Series Prediction of Viable Embryo and Automatic Grading in IVF using Deep Learning

Background:

The process of In Vitro Fertilization (IVF) involves collecting multiple samples of mature eggs that are fertilized with sperms in the IVF laboratory. They are eventually graded, and the most viable embryo out of all the samples is selected for transfer in the mother’s womb for a healthy pregnancy. Currently, the process of grading and selecting the healthiest embryo is performed by visual morphology, and manual records are maintained by embryologists.

Objectives:

Maintaining manual records makes the process very tedious, time-consuming, and error-prone. The absence of a universal grading leads to high subjectivity and low success rate of pregnancy. To improve the chances of pregnancy, multiple embryos are transferred in the womb elevating the risk of multiple pregnancies. In this paper, we propose a deep learning-based method to perform the automatic grading of the embryos using time series prediction with Long Short Term Memory (LSTM) and Convolutional Neural Network (CNN).

Methods:

CNN extracts the features of the images of embryos, and a sequence of such features is fed to LSTM for time series prediction, which gives the final grade.

Results:

Our model gave an ideal accuracy of 100% on training and validation. A comparison of obtained results is made with those obtained from a GRU model as well as other pre-trained models.

Conclusion:

The automated process is robust and eliminates subjectivity. The days-long hard work can now be replaced with our model, which gives the grading within 8 seconds with a GPU.

Artificial intelligence in the embryology laboratory: a review

The goal of an IVF cycle is a healthy live-born baby. Despite the many advances in the field of assisted reproductive technologies, accurately predicting the outcome of an IVF cycle has yet to be achieved. One reason for this is the method of selecting an embryo for transfer. Morphological assessment of embryos is the traditional method of evaluating embryo quality and selecting which embryo to transfer. However, this subjective method of assessing embryos leads to inter- and intra-observer variability, resulting in less than optimal IVF success rates. To overcome this, it is common practice to transfer more than one embryo, potentially resulting in high-risk multiple pregnancies. Although time-lapse incubators and preimplantation genetic testing for aneuploidy have been introduced to help increase the chances of live birth, the outcomes remain less than ideal. Utilization of artificial intelligence (AI) has become increasingly popular in the medical field and is increasingly being leveraged in the embryology laboratory to help improve IVF outcomes. Many studies have been published investigating the use of AI as an unbiased, automated approach to embryo assessment. This review summarizes recent AI advancements in the embryology laboratory.

Should there be an "AI" in TEAM? Embryologists selection of high implantation potential embryos improves with the aid of an artificial intelligence algorithm

PURPOSE

A deep learning artificial intelligence (AI) algorithm has been demonstrated to outperform embryologists in identifying euploid embryos destined to implant with an accuracy of 75.3% (1). Our aim was to evaluate the performance of highly trained embryologists in selecting top quality day 5 euploid blastocysts with and without the aid of a deep learning algorithm.

MATERIALS AND METHODS

A non-overlapping series of 200 sets of day 5 euploid embryo images with known implantation outcomes was distributed to 17 highly trained embryologists. One embryo in each set was known to have implanted and one failed implantation. They were asked to select which embryo to transfer from each set. The same 200 sets of embryos, with indication of which embryo in each set had been identified by the algorithm as more likely to implant was then distributed. Chi-squared, t-test, and receiver operating curves were performed to compare the embryologist performeance with and without AI.

RESULTS

Fourteen embryologists completed both assessments. Embryologists provided with AI results selected successfully implanted embryos in 73.6% of cases compared to 65.5% for those selected using visual assessments alone (p < 0.001). All embryologists improved in their ability to select embryos with the aid of the AI algorithm with a mean percent improvement of 11.1% (range 1.4% to 15.5%). There were no differences in degree of improvement by embryologist level of experience (junior, intermediate, senior).

CONCLUSIONS

The incorporation of an AI framework for blastocyst selection enhanced the performance of trained embryologists in identifying PGT-A euploid embryos destined to implant.

End-to-end deep learning for recognition of ploidy status using time-lapse videos

PURPOSE

Our retrospective study is to investigate an end-to-end deep learning model in identifying ploidy status through raw time-lapse video.

METHODS

By randomly dividing the dataset of time-lapse videos with known outcome of preimplantation genetic testing for aneuploidy (PGT-A), a deep learning model on raw videos was trained by the 80% dataset, and used to test the remaining 20%, by feeding time-lapse videos as input and the PGT-A prediction as output. The performance was measured by an average area under the curve (AUC) of the receiver operating characteristic curve.

RESULT(S)

With 690 sets of time-lapse video image, combined with PGT-A results, our deep learning model has achieved an AUC of 0.74 from the test dataset (138 videos), in discriminating between aneuploid embryos (group 1) and others (group 2, including euploid and mosaic embryos).

CONCLUSION

Our model demonstrated a proof of concept and potential in recognizing the ploidy status of tested embryos. A larger scale and further optimization on the exclusion criteria would be included in our future investigation, as well as prospective approach.

Embryo selection with artificial intelligence: how to evaluate and compare methods?

Embryo selection within in vitro fertilization (IVF) is the process of evaluating qualities of fertilized oocytes (embryos) and selecting the best embryo(s) available within a patient cohort for subsequent transfer or cryopreservation. In recent years, artificial intelligence (AI) has been used extensively to improve and automate the embryo ranking and selection procedure by extracting relevant information from embryo microscopy images. The AI models are evaluated based on their ability to identify the embryo(s) with the highest chance(s) of achieving a successful pregnancy. Whether such evaluations should be based on ranking performance or pregnancy prediction, however, seems to divide studies. As such, a variety of performance metrics are reported, and comparisons between studies are often made on different outcomes and data foundations. Moreover, superiority of AI methods over manual human evaluation is often claimed based on retrospective data, without any mentions of potential bias. In this paper, we provide a technical view on some of the major topics that divide how current AI models are trained, evaluated and compared. We explain and discuss the most common evaluation metrics and relate them to the two separate evaluation objectives, ranking and prediction. We also discuss when and how to compare AI models across studies and explain in detail how a selection bias is inevitable when comparing AI models against current embryo selection practice in retrospective cohort studies.

Artificial intelligence in human in vitro fertilization and embryology

Embryo evaluation and selection embody the aggregate manifestation of the entire in vitro fertilization (IVF) process. It aims to choose the "best" embryos from the larger cohort of fertilized oocytes, the majority of which will be determined to be not viable either as a result of abnormal development or due to chromosomal imbalances. Indeed, it is generally acknowledged that even after embryo selection based on morphology, time-lapse microscopic photography, or embryo biopsy with preimplantation genetic testing, implantation rates in the human are difficult to predict. Our pursuit of enhancing embryo evaluation and selection, as well as increasing live birth rates, will require the adoption of novel technologies. Recently, several artificial intelligence (AI)-based methods have emerged as objective, standardized, and efficient tools for evaluating human embryos. Moreover, AI-based methods can be implemented for other clinical aspects of IVF, such as assessing patient reproductive potential and individualizing gonadotropin stimulation protocols. As AI has the capability to analyze "big" data, the ultimate goal will be to apply AI tools to the analysis of all embryological, clinical, and genetic data in an effort to provide patient-tailored treatments. In this chapter, we present an overview of existing AI technologies in reproductive medicine and envision their potential future applications in the field.

Review of computer vision application in in vitro fertilization: the application of deep learning-based computer vision technology in the world of IVF

In vitro fertilization has been regarded as a forefront solution in treating infertility for over four decades, yet its effectiveness has remained relatively low. This could be attributed to the lack of advancements for the method of observing and selecting the most viable embryos for implantation. The conventional morphological assessment of embryos exhibits inevitable drawbacks which include time- and effort-consuming, and imminent risks of bias associated with subjective assessments performed by individual embryologists. A combination of these disadvantages, undeterred by the introduction of the time-lapse incubator technology, has been considered as a prominent contributor to the less preferable success rate of IVF cycles. Nonetheless, a recent surge of AI-based solutions for tasks automation in IVF has been observed. An AI-powered assistant could improve the efficiency of performing certain tasks in addition to offering accurate algorithms that behave as baselines to minimize the subjectivity of the decision-making process. Through a comprehensive review, we have discovered multiple approaches of implementing deep learning technology, each with varying degrees of success, for constructing the automated systems in IVF which could evaluate and even annotate the developmental stages of an embryo.

An artificial intelligence model based on the proteomic profile of euploid embryos and blastocyst morphology: a preliminary study

RESEARCH QUESTION

The study aimed to develop an artificial intelligence model based on artificial neural networks (ANNs) to predict the likelihood of achieving a live birth using the proteomic profile of spent culture media and blastocyst morphology.

DESIGN

This retrospective cohort study included 212 patients who underwent single blastocyst transfer at IVI Valencia. A single image of each of 186 embryos was studied, and the protein profile was analysed in 81 samples of spent embryo culture medium from patients included in the preimplantation genetic testing programme. The information extracted from the analyses was used as input data for the ANN. The multilayer perceptron and the back-propagation learning method were used to train the ANN. Finally, predictive power was measured using the area under the curve (AUC) of the receiver operating characteristic curve.

RESULTS

Three ANN architectures classified most of the embryos correctly as leading (LB+) or not leading (LB-) to a live birth: 100.0% for ANN1 (morphological variables and two proteins), 85.7% for ANN2 (morphological variables and seven proteins), and 83.3% for ANN3 (morphological variables and 25 proteins). The artificial intelligence model using information extracted from blastocyst image analysis and concentrations of interleukin-6 and matrix metalloproteinase-1 was able to predict live birth with an AUC of 1.0.

CONCLUSIONS

The model proposed in this preliminary report may provide a promising tool to select the embryo most likely to lead to a live birth in a euploid cohort. The accuracy of prediction demonstrated by this software may improve the efficacy of an assisted reproduction treatment by reducing the number of transfers per patient. Prospective studies are, however, needed.

Artificial intelligence in the IVF laboratory: overview through the application of different types of algorithms for the classification of reproductive data

Over the past years, the assisted reproductive technologies (ARTs) have been accompanied by constant innovations. For instance, intracytoplasmic sperm injection (ICSI), time-lapse monitoring of the embryonic morphokinetics, and PGS are innovative techniques that increased the success of the ART. In the same trend, the use of artificial intelligence (AI) techniques is being intensively researched whether in the embryo or spermatozoa selection. Despite several studies already published, the use of AI within assisted reproduction clinics is not yet a reality. This is largely due to the different AI techniques that are being proposed to be used in the daily routine of the clinics, which causes some uncertainty in their use. To shed light on this complex scenario, this review briefly describes some of the most frequently used AI algorithms, their functionalities, and their potential use. Several databases were analyzed in search of articles where applied artificial intelligence algorithms were used on reproductive data. Our focus was on the classification of embryonic cells and semen samples. Of a total of 124 articles analyzed, 32 were selected for this review. From the proposed algorithms, most have achieved a satisfactory precision, demonstrating the potential of a wide range of AI techniques. However, the evaluation of these studies suggests the need for more standardized research to validate the proposed models and their algorithms. Routine use of AI in assisted reproduction clinics is just a matter of time. However, the choice of AI technique to be used is supported by a better understanding of the principles subjacent to each technique, that is, its robustness, pros, and cons. We provide some current (although incipient) and potential uses of AI on the clinic routine, discussing how accurate and friendly it could be. Finally, we propose some standards for AI research on the selection of the embryo to be transferred and other future hints. For us, the imminence of its use is evident, providing a revolutionary milestone that will impact the ART.

Do patient factors influence embryologists' decisions to freeze borderline blastocysts?

RESEARCH QUESTION

To investigate whether patient factors influence the decision to freeze a blastocyst with low implantation potential.

DESIGN

This experimental study assessed 170 practicing embryologists from a variety of countries who were recruited via an online survey. Participants were currently practicing embryologists, who grade blastocysts as part of this role. The survey presented decision-making 'vignettes' to participants. These included specific patient information, as well as an image of an expanded blastocyst that was of borderline quality for inner cell mass and trophectoderm, for which the embryologist selected whether or not to freeze. High/low maternal age, the presence/absence of other top quality blastocysts, and the presence/absence of previously unsuccessful IVF cycles were systematically varied within the patient information in a 2 × 2 × 2 design. Participants reported how likely they would be to freeze a particular blastocyst on a scale of 1 (Extremely Unlikely) to 7 (Extremely Likely), and whether or not they would ultimately freeze each blastocyst (Yes or No).

RESULTS

Lower maternal age, no other high-quality blastocysts within the cohort, and multiple unsuccessful IVF cycles were associated with greater likelihood of recommending to freeze (P < .001). Furthermore, significant interactions among all three patient factors were noted.

CONCLUSION

This study provides evidence suggesting that when faced with an uncertain blastocyst, factors pertaining to the patient (maternal age, the presence/absence of other top quality blastocysts, and the presence/absence of previously unsuccessful IVF cycles) influence the decision to freeze.