Development of deep learning algorithms for predicting blastocyst formation and quality by time-lapse monitoring

Prediction of pregnancy-related complications in women undergoing assisted reproduction, using machine learning methods

OBJECTIVE

To use machine learning methods to develop prediction models of pregnancy complications in women who conceived with assisted reproductive techniques (ART).

DESIGN

A nation-wide register-based cohort study with prospectively collected data.

SETTING

Swedish national registers and nationwide quality IVF register.

PATIENT(S)

all nulliparous women who achieved birth within the first 3 ART treatment cycles between 2008 and 2016 in Sweden.

INTERVENTION(S)

Characteristics before the use of ART, such as demographics and medical history, were considered potential predictors in the development of before treatment prediction models. ART treatment details were further included in after treatment prediction models.

MAIN OUTCOME MEASURE(S)

Potential diagnoses of preeclampsia, placental complications (previa, accreta, and abruption), and postpartum hemorrhage were identified using the International Classification of Diseases recorded in the Swedish Medical Birth and Patient registers, respectively. Multiple prediction model algorithms were performed and compared for each outcome and treatment cycle, including logistic regression, decision tree model, naïve Bayes classification, support vector machine, random forest, and gradient boosting. The performance of each model was assessed with C statistic, and nested cross-validation was used to aid model selection and hyperparameter tuning.

RESULT(S)

A total of 14,732 women gave birth after the first (N = 7,302), second (N = 4,688), or third (N = 2,742) ART cycle, representing birth rates of 24.1%, 23.8%, and 22.0%. Overall prediction performance did not vary much across the different methods used. In the first cycle, the before treatment prediction performance was at best 66%, 66%, and 60% for preeclampsia, placental complications, and postpartum hemorrhage, respectively. Inclusion of after treatment characteristics conferred slight improvement (approximately 1%-5%), as did prediction in later cycles (approximately 1%-5%). The top influential and consistent predictors included age, region of residence, infertility diagnosis, and type of embryo transfer (fresh or frozen) in the later (2nd and 3rd) cycles. Body mass index was a top predictor of preeclampsia and was also influential for placental complications but not for postpartum hemorrhage.

CONCLUSION(S)

The combined use of demographics, medical history, and ART treatment information was not enough to confidently predict serious pregnancy complications in women who conceived with ART. Future studies are needed to assess if additional longitudinal follow-up during pregnancy can improve the prediction to allow clinical protocol development.

Artificial intelligence-powered assisted ranking of sibling embryos to increase first cycle pregnancy rate

RESEARCH QUESTION

Could EMBRYOLY, an artificial intelligence embryo evaluation tool, assist embryologists to increase first cycle pregnancy rate and reduce cycles to pregnancy for patients?

DESIGN

Data from 11,988 embryos were collected via EMBRYOLY from 2666 egg retrievals (2019-2022) across 11 centres in France, Spain and Morocco using three time-lapse systems (TLS). Data from two independent clinics were also examined. EMBRYOLY's transformer-based model was applied to transferred embryos to evaluate ranking performances against pregnancy and birth outcomes. It was applied to cohorts to rank sibling embryos (including non-transferred) according to their likelihood of clinical pregnancy and to compute the agreement with the embryologist's highest ranked embryo. Its effect on time to pregnancy and first cycle pregnancy rate was evaluated on cohorts with multiple single blastocyst transfers, assuming the embryologist would have considered EMBRYOLY's ranking on the embryos favoured for transfer.

RESULTS

EMBRYOLY's score correlated significantly with clinical pregnancies and live births for cleavage and blastocyst transfers. This held true for clinical pregnancies from blastocyst transfers in two independent clinics. In cases of multiple single embryo transfers, embryologists achieved a 19.8% first cycle pregnancy rate, which could have been improved to 44.1% with the adjunctive use of EMBRYOLY (McNemar's test: P < 0.001). This could have reduced cycles to clinical pregnancy from 2.01 to 1.66 (Wilcoxon test: P < 0.001).

CONCLUSIONS

EMBRYOLY's potential to enhance first cycle pregnancy rates when combined with embryologists' expertise is highlighted. It reduces the number of unsuccessful cycles for patients across TLS and IVF centres.

Viewing early life without labels: optical approaches for imaging the early embryo†

Embryo quality is an important determinant of successful implantation and a resultant live birth. Current clinical approaches for evaluating embryo quality rely on subjective morphology assessments or an invasive biopsy for genetic testing. However, both approaches can be inherently inaccurate and crucially, fail to improve the live birth rate following the transfer of in vitro produced embryos. Optical imaging offers a potential non-invasive and accurate avenue for assessing embryo viability. Recent advances in various label-free optical imaging approaches have garnered increased interest in the field of reproductive biology due to their ability to rapidly capture images at high resolution, delivering both morphological and molecular information. This burgeoning field holds immense potential for further development, with profound implications for clinical translation. Here, our review aims to: (1) describe the principles of various imaging systems, distinguishing between approaches that capture morphological and molecular information, (2) highlight the recent application of these technologies in the field of reproductive biology, and (3) assess their respective merits and limitations concerning the capacity to evaluate embryo quality. Additionally, the review summarizes challenges in the translation of optical imaging systems into routine clinical practice, providing recommendations for their future development. Finally, we identify suitable imaging approaches for interrogating the mechanisms underpinning successful embryo development.

An artificial intelligence algorithm to select most viable embryos considering current process in IVF labs

Background

The most common Assisted Reproductive Technology is In-Vitro Fertilization (IVF). During IVF, embryologists commonly perform a morphological assessment to evaluate embryo quality and choose the best embryo for transferring to the uterus. However, embryo selection through morphological assessment is subjective, so various embryologists obtain different conclusions. Furthermore, humans can consider only a limited number of visual parameters resulting in a poor IVF success rate. Artificial intelligence (AI) for embryo selection is objective and can include many parameters, leading to better IVF outcomes.

Objectives

This study sought to use AI to (1) predict pregnancy results based on embryo images, (2) assess using more than one image of the embryo in the prediction of pregnancy but based on the current process in IVF labs, and (3) compare results of AI-Based methods and embryologist experts in predicting pregnancy.

Methods

A data set including 252 Time-lapse Videos of embryos related to IVF performed between 2017 and 2020 was collected. Frames related to 19 ± 1, 43 ± 1, and 67 ± 1 h post-insemination were extracted. Well-Known CNN architectures with transfer learning have been applied to these images. The results have been compared with an algorithm that only uses the final image of embryos. Furthermore, the results have been compared with five experienced embryologists.

Results

To predict the pregnancy outcome, we applied five well-known CNN architectures (AlexNet, ResNet18, ResNet34, Inception V3, and DenseNet121). DeepEmbryo, using three images, predicts pregnancy better than the algorithm that only uses one final image. It also can predict pregnancy better than all embryologists. Different well-known architectures can successfully predict pregnancy chances with up to 75.0% accuracy using Transfer Learning.

Conclusion

We have developed DeepEmbryo, an AI-based tool that uses three static images to predict pregnancy. Additionally, DeepEmbryo uses images that can be obtained in the current IVF process in almost all IVF labs. AI-based tools have great potential for predicting pregnancy and can be used as a proper tool in the future.

A clinical consensus-compliant deep learning approach to quantitatively evaluate human in vitro fertilization early embryonic development with optical microscope images

The selection of embryos is a key for the success of in vitro fertilization (IVF). However, automatic quality assessment on human IVF embryos with optical microscope images is still challenging. In this study, we developed a clinical consensus-compliant deep learning approach, named Esava (Embryo Segmentation and Viability Assessment), to quantitatively evaluate the development of IVF embryos using optical microscope images. In total 551 optical microscope images of human IVF embryos of day-2 to day-3 were collected, preprocessed, and annotated. Using the Faster R-CNN model as baseline, our Esava model was constructed, refined, trained, and validated for precise and robust blastomere detection. A novel algorithm Crowd-NMS was proposed and employed in Esava to enhance the object detection and to precisely quantify the embryonic cells and their size uniformity. Additionally, an innovative GrabCut-based unsupervised module was integrated for the segmentation of blastomeres and embryos. Independently tested on 94 embryo images for blastomere detection, Esava obtained the high rates of 0.9940, 0.9121, and 0.9531 for precision, recall, and mAP respectively, and gained significant advances compared with previous computational methods. Intraclass correlation coefficients indicated the consistency between Esava and three experienced embryologists. Another test on 51 extra images demonstrated that Esava surpassed other tools significantly, achieving the highest average precision 0.9025. Moreover, it also accurately identified the borders of blastomeres with mIoU over 0.88 on the independent testing dataset. Esava is compliant with the Istanbul clinical consensus and compatible to senior embryologists. Taken together, Esava improves the accuracy and efficiency of embryonic development assessment with optical microscope images.

Criteria for implementing artificial intelligence systems in reproductive medicine

This review article discusses the integration of artificial intelligence (AI) in assisted reproductive technology and provides key concepts to consider when introducing AI systems into reproductive medicine practices. The article highlights the various applications of AI in reproductive medicine and discusses whether to use commercial or in-house AI systems. This review also provides criteria for implementing new AI systems in the laboratory and discusses the factors that should be considered when introducing AI in the laboratory, including the user interface, scalability, training, support, follow-up, cost, ethics, and data quality. The article emphasises the importance of ethical considerations, data quality, and continuous algorithm updates to ensure the accuracy and safety of AI systems.

Artificial intelligence in time-lapse system: advances, applications, and future perspectives in reproductive medicine

With the rising demand for in vitro fertilization (IVF) cycles, there is a growing need for innovative techniques to optimize procedure outcomes. One such technique is time-lapse system (TLS) for embryo incubation, which minimizes environmental changes in the embryo culture process. TLS also significantly advances predicting embryo quality, a crucial determinant of IVF cycle success. However, the current subjective nature of embryo assessments is due to inter- and intra-observer subjectivity, resulting in highly variable results. To address this challenge, reproductive medicine has gradually turned to artificial intelligence (AI) to establish a standardized and objective approach, aiming to achieve higher success rates. Extensive research is underway investigating the utilization of AI in TLS to predict multiple outcomes. These studies explore the application of popular AI algorithms, their specific implementations, and the achieved advancements in TLS. This review aims to provide an overview of the advances in AI algorithms and their particular applications within the context of TLS and the potential challenges and opportunities for further advancements in reproductive medicine.

Interpretable artificial intelligence-assisted embryo selection improved single-blastocyst transfer outcomes: a prospective cohort study

RESEARCH QUESTION

What is the pregnancy and neonatal outcomes of an interpretable artificial intelligence (AI) model for embryo selection in a prospective clinical trial?

DESIGN

This single-centre prospective cohort study was carried out from October 2021 to March 2022. A total of 330 eligible patients were assigned to their preferred groups, with 250 patients undergoing a fresh single-blastocyst transfer cycle after the exclusion criteria had been applied. For the AI-assisted group (AAG), embryologists selected the embryos for transfer based on the ranking recommendations provided by an interpretable AI system, while with the manual group, embryologists used the Gardner grading system to make their decisions.

RESULTS

The implantation rate was significantly higher in the AAG than the manual group (80.87% versus 68.15%, P = 0.022). No significant difference was found in terms of monozygotic twin rate, miscarriage rate, live birth rate and ectopic pregnancy rate between the groups. Furthermore, there was no significant difference in terms of neonatal outcomes, including gestational weeks, premature birth rate, birth height, birthweight, sex ratio at birth and newborn malformation rate. The consensus rate between the AI and retrospective analysis by the embryologists was significantly higher for good-quality embryos (i.e. grade 4BB or higher) versus poor-quality embryos (i.e. less than 4BB) (84.71% versus 25%, P < 0.001).

CONCLUSIONS

These prospective trial results suggest that the proposed AI system could effectively help embryologists to improve the implantation rate with single-blastocyst transfer compared with traditional manual evaluation methods.

Morphology-based deep learning approach for predicting adipogenic and osteogenic differentiation of human mesenchymal stem cells (hMSCs)

Human mesenchymal stem cells (hMSCs) are multipotent progenitor cells with the potential to differentiate into various cell types, including osteoblasts, chondrocytes, and adipocytes. These cells have been extensively employed in the field of cell-based therapies and regenerative medicine due to their inherent attributes of self-renewal and multipotency. Traditional approaches for assessing hMSCs differentiation capacity have relied heavily on labor-intensive techniques, such as RT-PCR, immunostaining, and Western blot, to identify specific biomarkers. However, these methods are not only time-consuming and economically demanding, but also require the fixation of cells, resulting in the loss of temporal data. Consequently, there is an emerging need for a more efficient and precise approach to predict hMSCs differentiation in live cells, particularly for osteogenic and adipogenic differentiation. In response to this need, we developed innovative approaches that combine live-cell imaging with cutting-edge deep learning techniques, specifically employing a convolutional neural network (CNN) to meticulously classify osteogenic and adipogenic differentiation. Specifically, four notable pre-trained CNN models, VGG 19, Inception V3, ResNet 18, and ResNet 50, were developed and tested for identifying adipogenic and osteogenic differentiated cells based on cell morphology changes. We rigorously evaluated the performance of these four models concerning binary and multi-class classification of differentiated cells at various time intervals, focusing on pivotal metrics such as accuracy, the area under the receiver operating characteristic curve (AUC), sensitivity, precision, and F1-score. Among these four different models, ResNet 50 has proven to be the most effective choice with the highest accuracy (0.9572 for binary, 0.9474 for multi-class) and AUC (0.9958 for binary, 0.9836 for multi-class) in both multi-class and binary classification tasks. Although VGG 19 matched the accuracy of ResNet 50 in both tasks, ResNet 50 consistently outperformed it in terms of AUC, underscoring its superior effectiveness in identifying differentiated cells. Overall, our study demonstrated the capability to use a CNN approach to predict stem cell fate based on morphology changes, which will potentially provide insights for the application of cell-based therapy and advance our understanding of regenerative medicine.

Deep learning for embryo evaluation using time-lapse: a systematic review of diagnostic test accuracy

OBJECTIVE

This study aimed to investigate the accuracy of convolutional neural network models in the assessment of embryos using time-lapse monitoring.

DATA SOURCES

A systematic search was conducted in PubMed and Web of Science databases from January 2016 to December 2022. The search strategy was carried out by using key words and MeSH (Medical Subject Headings) terms.

STUDY ELIGIBILITY CRITERIA

Studies were included if they reported the accuracy of convolutional neural network models for embryo evaluation using time-lapse monitoring. The review was registered with PROSPERO (International Prospective Register of Systematic Reviews; identification number CRD42021275916).

METHODS

Two reviewer authors independently screened results using the Covidence systematic review software. The full-text articles were reviewed when studies met the inclusion criteria or in any uncertainty. Nonconsensus was resolved by a third reviewer. Risk of bias and applicability were evaluated using the QUADAS-2 tool and the modified Joanna Briggs Institute or JBI checklist.

RESULTS

Following a systematic search of the literature, 22 studies were identified as eligible for inclusion. All studies were retrospective. A total of 522,516 images of 222,998 embryos were analyzed. Three main outcomes were evaluated: successful in vitro fertilization, blastocyst stage classification, and blastocyst quality. Most studies reported >80% accuracy, and embryologists were outperformed in some. Ten studies had a high risk of bias, mostly because of patient bias.

CONCLUSION

The application of artificial intelligence in time-lapse monitoring has the potential to provide more efficient, accurate, and objective embryo evaluation. Models that examined blastocyst stage classification showed the best predictions. Models that predicted live birth had a low risk of bias, used the largest databases, and had external validation, which heightens their relevance to clinical application. Our systematic review is limited by the high heterogeneity among the included studies. Researchers should share databases and standardize reporting.

Application of artificial intelligence in gametes and embryos selection

Gamete and embryo quality are critical to the success rate of Assisted Reproductive Technology (ART) cycles, but there remains a lack of methods to accurately measure the quality of sperm, oocytes and embryos. The ability of Artificial Intelligence (AI) technology to analyze large amounts of data, especially video and images, is particularly useful in gamete and embryo assessment and selection. The well-trained model has fast calculation speed and high accuracy, which can help embryologists to perform more objective gamete and embryo selection. Various artificial intelligence models have been developed for gamete and embryo assessment, some of which exhibit good performance. In this review, we summarize the latest applications of AI technology in semen analysis, as well as selection for sperm, oocyte and embryo, and discuss the existing problems and development directions of artificial intelligence in this field.

The embryo non-invasive pre-implantation diagnosis era: how far are we?

Advancements in assisted reproduction (AR) methodologies have allowed significant improvements in live birth rates of women who otherwise would not be able to conceive. One of the tools that allowed this improvement is the possibility of embryo selection based on genetic status, performed via preimplantation genetic testing (PGT). Even though the widespread use of PGT from TE biopsy helped to decrease the interval from the beginning of the AR intervention to pregnancy, especially in older patients, in AR, there are still many concerns about the application of this invasive methodology in all cycles. Therefore, recently, researchers started to study the use of cell free DNA (cfDNA) released by the blastocyst in its culture medium to perform PGT, in a method called non-invasive PGT (niPGT). The development of a niPGT would bring the diagnostics power of conventional PGT, but with the advantage of being potentially less harmful to the embryo. Its implementation in clinical practice, however, is under heavy discussion since there are many unknowns about the technique, such as the origin of the cfDNA or if this genetic material is a true representative of the actual ploidy status of the embryo. Available data indicates that there is high correspondence between results observed in TE biopsies and the ones observed from cfDNA, but these results are still contradictory and highly debatable. In the present review, the advantages and disadvantages of niPGT are presented and discussed in relation to tradition TE biopsy-based PGT. Furthermore, there are also presented some other possible non-invasive tools that could be applied in the selection of the best embryo, such as quantification of other molecules as quality biomarkers, or the use artificial intelligence (AI) to identify the best embryos based on morphological and/or morphokitetic parameters.

Embryo selection through artificial intelligence versus embryologists: a systematic review

STUDY QUESTION

What is the present performance of artificial intelligence (AI) decision support during embryo selection compared to the standard embryo selection by embryologists?

SUMMARY ANSWER

AI consistently outperformed the clinical teams in all the studies focused on embryo morphology and clinical outcome prediction during embryo selection assessment.

WHAT IS KNOWN ALREADY

The ART success rate is ∼30%, with a worrying trend of increasing female age correlating with considerably worse results. As such, there have been ongoing efforts to address this low success rate through the development of new technologies. With the advent of AI, there is potential for machine learning to be applied in such a manner that areas limited by human subjectivity, such as embryo selection, can be enhanced through increased objectivity. Given the potential of AI to improve IVF success rates, it remains crucial to review the performance between AI and embryologists during embryo selection.

STUDY DESIGN SIZE DURATION

The search was done across PubMed, EMBASE, Ovid Medline, and IEEE Xplore from 1 June 2005 up to and including 7 January 2022. Included articles were also restricted to those written in English. Search terms utilized across all databases for the study were: ('Artificial intelligence' OR 'Machine Learning' OR 'Deep learning' OR 'Neural network') AND ('IVF' OR 'in vitro fertili*' OR 'assisted reproductive techn*' OR 'embryo'), where the character '*' refers the search engine to include any auto completion of the search term.

PARTICIPANTS/MATERIALS SETTING METHODS

A literature search was conducted for literature relating to AI applications to IVF. Primary outcomes of interest were accuracy, sensitivity, and specificity of the embryo morphology grade assessments and the likelihood of clinical outcomes, such as clinical pregnancy after IVF treatments. Risk of bias was assessed using the Modified Down and Black Checklist.

MAIN RESULTS AND THE ROLE OF CHANCE

Twenty articles were included in this review. There was no specific embryo assessment day across the studies-Day 1 until Day 5/6 of embryo development was investigated. The types of input for training AI algorithms were images and time-lapse (10/20), clinical information (6/20), and both images and clinical information (4/20). Each AI model demonstrated promise when compared to an embryologist's visual assessment. On average, the models predicted the likelihood of successful clinical pregnancy with greater accuracy than clinical embryologists, signifying greater reliability when compared to human prediction. The AI models performed at a median accuracy of 75.5% (range 59-94%) on predicting embryo morphology grade. The correct prediction (Ground Truth) was defined through the use of embryo images according to post embryologists' assessment following local respective guidelines. Using blind test datasets, the embryologists' accuracy prediction was 65.4% (range 47-75%) with the same ground truth provided by the original local respective assessment. Similarly, AI models had a median accuracy of 77.8% (range 68-90%) in predicting clinical pregnancy through the use of patient clinical treatment information compared to 64% (range 58-76%) when performed by embryologists. When both images/time-lapse and clinical information inputs were combined, the median accuracy by the AI models was higher at 81.5% (range 67-98%), while clinical embryologists had a median accuracy of 51% (range 43-59%).

LIMITATIONS REASONS FOR CAUTION

The findings of this review are based on studies that have not been prospectively evaluated in a clinical setting. Additionally, a fair comparison of all the studies were deemed unfeasible owing to the heterogeneity of the studies, development of the AI models, database employed and the study design and quality.

WIDER IMPLICATIONS OF THE FINDINGS

AI provides considerable promise to the IVF field and embryo selection. However, there needs to be a shift in developers' perception of the clinical outcome from successful implantation towards ongoing pregnancy or live birth. Additionally, existing models focus on locally generated databases and many lack external validation.

STUDY FUNDING/COMPETING INTERESTS

This study was funded by Monash Data Future Institute. All authors have no conflicts of interest to declare.

REGISTRATION NUMBER

CRD42021256333.

Towards Automation in IVF: Pre-Clinical Validation of a Deep Learning-Based Embryo Grading System during PGT-A Cycles

Preimplantation genetic testing for aneuploidies (PGT-A) is arguably the most effective embryo selection strategy. Nevertheless, it requires greater workload, costs, and expertise. Therefore, a quest towards user-friendly, non-invasive strategies is ongoing. Although insufficient to replace PGT-A, embryo morphological evaluation is significantly associated with embryonic competence, but scarcely reproducible. Recently, artificial intelligence-powered analyses have been proposed to objectify and automate image evaluations. iDAScore v1.0 is a deep-learning model based on a 3D convolutional neural network trained on time-lapse videos from implanted and non-implanted blastocysts. It is a decision support system for ranking blastocysts without manual input. This retrospective, pre-clinical, external validation included 3604 blastocysts and 808 euploid transfers from 1232 cycles. All blastocysts were retrospectively assessed through the iDAScore v1.0; therefore, it did not influence embryologists' decision-making process. iDAScore v1.0 was significantly associated with embryo morphology and competence, although AUCs for euploidy and live-birth prediction were 0.60 and 0.66, respectively, which is rather comparable to embryologists' performance. Nevertheless, iDAScore v1.0 is objective and reproducible, while embryologists' evaluations are not. In a retrospective simulation, iDAScore v1.0 would have ranked euploid blastocysts as top quality in 63% of cases with one or more euploid and aneuploid blastocysts, and it would have questioned embryologists' ranking in 48% of cases with two or more euploid blastocysts and one or more live birth. Therefore, iDAScore v1.0 may objectify embryologists' evaluations, but randomized controlled trials are required to assess its clinical value.

A hybrid artificial intelligence model leverages multi-centric clinical data to improve fetal heart rate pregnancy prediction across time-lapse systems

STUDY QUESTION

Can artificial intelligence (AI) algorithms developed to assist embryologists in evaluating embryo morphokinetics be enriched with multi-centric clinical data to better predict clinical pregnancy outcome?

SUMMARY ANSWER

Training algorithms on multi-centric clinical data significantly increased AUC compared to algorithms that only analyzed the time-lapse system (TLS) videos.

WHAT IS KNOWN ALREADY

Several AI-based algorithms have been developed to predict pregnancy, most of them based only on analysis of the time-lapse recording of embryo development. It remains unclear, however, whether considering numerous clinical features can improve the predictive performances of time-lapse based embryo evaluation.

STUDY DESIGN, SIZE, DURATION

A dataset of 9986 embryos (95.60% known clinical pregnancy outcome, 32.47% frozen transfers) from 5226 patients from 14 European fertility centers (in two countries) recorded with three different TLS was used to train and validate the algorithms. A total of 31 clinical factors were collected. A separate test set (447 videos) was used to compare performances between embryologists and the algorithm.

PARTICIPANTS/MATERIALS, SETTING, METHODS

Clinical pregnancy (defined as a pregnancy leading to a fetal heartbeat) outcome was first predicted using a 3D convolutional neural network that analyzed videos of the embryonic development up to 2 or 3 days of development (33% of the database) or up to 5 or 6 days of development (67% of the database). The output video score was then fed as input alongside clinical features to a gradient boosting algorithm that generated a second score corresponding to the hybrid model. AUC was computed across 7-fold of the validation dataset for both models. These predictions were compared to those of 13 senior embryologists made on the test dataset.

MAIN RESULTS AND THE ROLE OF CHANCE

The average AUC of the hybrid model across all 7-fold was significantly higher than that of the video model (0.727 versus 0.684, respectively, P = 0.015; Wilcoxon test). A SHapley Additive exPlanations (SHAP) analysis of the hybrid model showed that the six first most important features to predict pregnancy were morphokinetics of the embryo (video score), oocyte age, total gonadotrophin dose intake, number of embryos generated, number of oocytes retrieved, and endometrium thickness. The hybrid model was shown to be superior to embryologists with respect to different metrics, including the balanced accuracy (P ≤ 0.003; Wilcoxon test). The likelihood of pregnancy was linearly linked to the hybrid score, with increasing odds ratio (maximum P-value = 0.001), demonstrating the ranking capacity of the model. Training individual hybrid models did not improve predictive performance. A clinic hold-out experiment was conducted and resulted in AUCs ranging between 0.63 and 0.73. Performance of the hybrid model did not vary between TLS or between subgroups of embryos transferred at different days of embryonic development. The hybrid model did fare better for patients older than 35 years (P < 0.001; Mann-Whitney test), and for fresh transfers (P < 0.001; Mann-Whitney test).

LIMITATIONS, REASONS FOR CAUTION

Participant centers were located in two countries, thus limiting the generalization of our conclusion to wider subpopulations of patients. Not all clinical features were available for all embryos, thus limiting the performances of the hybrid model in some instances.

WIDER IMPLICATIONS OF THE FINDINGS

Our study suggests that considering clinical data improves pregnancy predictive performances and that there is no need to retrain algorithms at the clinic level unless they follow strikingly different practices. This study characterizes a versatile AI algorithm with similar performance on different time-lapse microscopes and on embryos transferred at different development stages. It can also help with patients of different ages and protocols used but with varying performances, presumably because the task of predicting fetal heartbeat becomes more or less hard depending on the clinical context. This AI model can be made widely available and can help embryologists in a wide range of clinical scenarios to standardize their practices.

STUDY FUNDING/COMPETING INTEREST(S)

Funding for the study was provided by ImVitro with grant funding received in part from BPIFrance (Bourse French Tech Emergence (DOS0106572/00), Paris Innovation Amorçage (DOS0132841/00), and Aide au Développement DeepTech (DOS0152872/00)). A.B.-C. is a co-owner of, and holds stocks in, ImVitro SAS. A.B.-C. and F.D.M. hold a patent for 'Devices and processes for machine learning prediction of in vitro fertilization' (EP20305914.2). A.D., N.D., M.M.F., and F.D.M. are or have been employees of ImVitro and have been granted stock options. X.P.-V. has been paid as a consultant to ImVitro and has been granted stocks options of ImVitro. L.C.-D. and C.G.-S. have undertaken paid consultancy for ImVitro SAS. The remaining authors have no conflicts to declare.

TRIAL REGISTRATION NUMBER

N/A.

An explainable deep learning-based algorithm with an attention mechanism for predicting the live birth potential of mouse embryos

In assisted reproductive technology (ART), embryos produced by in vitro fertilization (IVF) are graded according to their live birth potential, and high-grade embryos are preferentially transplanted. However, rates of live birth following clinical ART remain low worldwide. Grading is based on the embryo shape at a limited number of stages and does not consider the shape of embryos and intracellular structures, e.g., nuclei, at various stages important for normal embryogenesis. Here, we developed a Normalized Multi-View Attention Network (NVAN) that directly predicts live birth potential from the nuclear structure in live-cell fluorescence images of mouse embryos from zygote to across a wide range of stages. The input is morphological features of cell nuclei, which were extracted as multivariate time-series data by using the segmentation algorithm for mouse embryos. The classification accuracy of our method (83.87%) greatly exceeded that of existing machine-learning methods and that of visual inspection by embryo culture specialists. Our method also has a new attention mechanism that allows us to determine which values of multivariate time-series data, used to describe nuclear morphology, were the basis for the prediction. By visualizing the features that contributed most to the prediction of live birth potential, we found that the size and shape of the nucleus at the morula stage and at the time of cell division were important for live birth prediction. We anticipate that our method will help ART and developmental engineering as a new basic technology for IVF embryo selection.

Machine learning-based detection of label-free cancer stem-like cell fate

The detection of cancer stem-like cells (CSCs) is mainly based on molecular markers or functional tests giving a posteriori results. Therefore label-free and real-time detection of single CSCs remains a difficult challenge. The recent development of microfluidics has made it possible to perform high-throughput single cell imaging under controlled conditions and geometries. Such a throughput requires adapted image analysis pipelines while providing the necessary amount of data for the development of machine-learning algorithms. In this paper, we provide a data-driven study to assess the complexity of brightfield time-lapses to monitor the fate of isolated cancer stem-like cells in non-adherent conditions. We combined for the first time individual cell fate and cell state temporality analysis in a unique algorithm. We show that with our experimental system and on two different primary cell lines our optimized deep learning based algorithm outperforms classical computer vision and shallow learning-based algorithms in terms of accuracy while being faster than cutting-edge convolutional neural network (CNNs). With this study, we show that tailoring our deep learning-based algorithm to the image analysis problem yields better results than pre-trained models. As a result, such a rapid and accurate CNN is compatible with the rise of high-throughput data generation and opens the door to on-the-fly CSC fate analysis.

Current trends in artificial intelligence in reproductive endocrinology

PURPOSE OF REVIEW

Artificial Intelligence, a tool that integrates computer science and machine learning to mimic human decision-making processes, is transforming the world and changing the way we live. Recently, the healthcare industry has gradually adopted artificial intelligence in many applications and obtained some degree of success. In this review, we summarize the current applications of artificial intelligence in Reproductive Endocrinology, in both laboratory and clinical settings.

RECENT FINDINGS

Artificial Intelligence has been used to select the embryos with high implantation potential, proper ploidy status, to predict later embryo development, and to increase pregnancy and live birth rates. Some studies also suggested that artificial intelligence can help improve infertility diagnosis and patient management. Recently, it has been demonstrated that artificial intelligence also plays a role in effective laboratory quality control and performance.

SUMMARY

In this review, we discuss various applications of artificial intelligence in different areas of reproductive medicine. We summarize the current findings with their potentials and limitations, and also discuss the future direction for research and clinical applications.

Unique Deep Radiomic Signature Shows NMN Treatment Reverses Morphology of Oocytes from Aged Mice

The purpose of this study is to develop a deep radiomic signature based on an artificial intelligence (AI) model. This radiomic signature identifies oocyte morphological changes corresponding to reproductive aging in bright field images captured by optical light microscopy. Oocytes were collected from three mice groups: young (4- to 5-week-old) C57BL/6J female mice, aged (12-month-old) mice, and aged mice treated with the NAD+ precursor nicotinamide mononucleotide (NMN), a treatment recently shown to rejuvenate aspects of fertility in aged mice. We applied deep learning, swarm intelligence, and discriminative analysis to images of mouse oocytes taken by bright field microscopy to identify a highly informative deep radiomic signature (DRS) of oocyte morphology. Predictive DRS accuracy was determined by evaluating sensitivity, specificity, and cross-validation, and was visualized using scatter plots of the data associated with three groups: Young, old and Old + NMN. DRS could successfully distinguish morphological changes in oocytes associated with maternal age with 92% accuracy (AUC~1), reflecting this decline in oocyte quality. We then employed the DRS to evaluate the impact of the treatment of reproductively aged mice with NMN. The DRS signature classified 60% of oocytes from NMN-treated aged mice as having a ‘young’ morphology. In conclusion, the DRS signature developed in this study was successfully able to detect aging-related oocyte morphological changes. The significance of our approach is that DRS applied to bright field oocyte images will allow us to distinguish and select oocytes originally affected by reproductive aging and whose quality has been successfully restored by the NMN therapy.

Reporting on the Value of Artificial Intelligence in Predicting the Optimal Embryo for Transfer: A Systematic Review including Data Synthesis

Artificial intelligence (AI) has been gaining support in the field of in vitro fertilization (IVF). Despite the promising existing data, AI cannot yet claim gold-standard status, which serves as the rationale for this study. This systematic review and data synthesis aims to evaluate and report on the predictive capabilities of AI-based prediction models regarding IVF outcome. The study has been registered in PROSPERO (CRD42021242097). Following a systematic search of the literature in Pubmed/Medline, Embase, and Cochrane Central Library, 18 studies were identified as eligible for inclusion. Regarding live-birth, the Area Under the Curve (AUC) of the Summary Receiver Operating Characteristics (SROC) was 0.905, while the partial AUC (pAUC) was 0.755. The Observed: Expected ratio was 1.12 (95%CI: 0.26–2.37; 95%PI: 0.02–6.54). Regarding clinical pregnancy with fetal heartbeat, the AUC of the SROC was 0.722, while the pAUC was 0.774. The O:E ratio was 0.77 (95%CI: 0.54–1.05; 95%PI: 0.21–1.62). According to this data synthesis, the majority of the AI-based prediction models are successful in accurately predicting the IVF outcome regarding live birth, clinical pregnancy, clinical pregnancy with fetal heartbeat, and ploidy status. This review attempted to compare between AI and human prediction capabilities, and although studies do not allow for a meta-analysis, this systematic review indicates that the AI-based prediction models perform rather similarly to the embryologists’ evaluations. While AI models appear marginally more effective, they still have some way to go before they can claim to significantly surpass the clinical embryologists’ predictive competence.

In vitro investigation of mammalian peri-implantation embryogenesis†

The embryos attach and invade into the uterus and establish the connection with their mother in peri-implantation development. During this period, the pluripotent epiblast cells of embryo undergo symmetry breaking, cell lineage allocation, and morphogenetic remodeling, accompanying with the dramatic changes of transcriptome, epigenome, and signal pathways, to prepare a state for their differentiation and gastrulation. The progresses in mouse genetics and stem cell biology have largely advanced the knowledge of these transformations which are largely hindered by the hard accessibility of natural embryos. To gain insight into mammalian peri-implantation development, great efforts have been made in the field. Recently, the advances in the prolonged in vitro culture of blastocysts, the derivation of multiple pluripotent stem cells, as well as the construction of stem cell-based embryo-like models have opened novel avenues to investigate peri-implantation development in mammals, especially for the humans. Combining with other emerging new technologies, these new models will substantially promote the comprehension of mammalian peri-implantation development, accelerating the progress of reproductive and regenerative medicine.

Medical professional enhancement using explainable artificial intelligence in fetal cardiac ultrasound screening (supplementary)

Diagnostic support tools based on artificial intelligence (AI) have exhibited high performance in various medical fields. However, their clinical application remains challenging because of the lack of explanatory power in AI decisions (black box problem), making it difficult to build trust with medical professionals. Nevertheless, visualizing the internal representation of deep neural networks will increase explanatory power and improve the confidence of medical professionals in AI decisions. We propose a novel deep learning-based explainable representation “graph chart diagram” to support fetal cardiac ultrasound screening, which has low detection rates of congenital heart diseases due to the difficulty in mastering the technique. Screening performance improves using this representation from 0.966 to 0.975 for experts, 0.829 to 0.890 for fellows, and 0.616 to 0.748 for residents in the arithmetic mean of area under the curve of a receiver operating characteristic curve. This is the first demonstration wherein examiners used deep learning-based explainable representation to improve the performance of fetal cardiac ultrasound screening, highlighting the potential of explainable AI to augment examiner capabilities.

Pseudo contrastive labeling for predicting IVF embryo developmental potential

In vitro fertilization is typically associated with high failure rates per transfer, leading to an acute need for the identification of embryos with high developmental potential. Current methods are tailored to specific times after fertilization, often require expert inspection, and have low predictive power. Automatic methods are challenged by ambiguous labels, clinical heterogeneity, and the inability to utilize multiple developmental points. In this work, we propose a novel method that trains a classifier conditioned on the time since fertilization. This classifier is then integrated over time and its output is used to assign soft labels to pairs of samples. The classifier obtained by training on these soft labels presents a significant improvement in accuracy, even as early as 30 h post-fertilization. By integrating the classification scores, the predictive power is further improved. Our results are superior to previously reported methods, including the commercial KIDScore-D3 system, and a group of eight senior professionals, in classifying multiple groups of favorable embryos into groups defined as less favorable based on implantation outcomes, expert decisions based on developmental trajectories, and/or genetic tests.

Human Oocyte Morphology and Outcomes of Infertility Treatment: a Systematic Review

Oocyte morphology assessment is easy to implement in any laboratory with possible quality grading prior to fertilization. At present, comprehensive oocyte morphology scoring is not performed as a routine procedure. However, it may augment chances for successful treatment outcomes if a correlation with certain dysmorphisms can be proven. In order to determine a correlation between oocyte morphology and treatment outcome, we performed a systematic search in PubMed and Cochrane Controlled Trials Register following PRISMA guidelines. A total of 52 articles out of 6,755 search results met the inclusion criteria. Dark colour of the cytoplasm (observed with an incidence rate of 7%), homogeneous granularity of the cytoplasm (19%) and ovoid shape of oocytes (7%) appeared to have no influence on treatment outcome. Abnormalities such as refractile bodies (10%), fragmented first polar body (37%), dark zona pellucida (9%), enlarged perivitelline space (18%) and debris in it (21%) are likely to affect the treatment outcome to some extent. Finally, cytoplasmic vacuoles (4%), centrally located cytoplasmic granularity (12%) and clusters of smooth endoplasmic reticulum (4%) negatively impact infertility treatment outcomes. Nonetheless, morphological assessment is informative rather than predictive. Adding oocyte morphology to the artificial intelligence (AI)-driven selection process may improve the precision of the algorithms. Oocyte morphology assessment can be especially useful in oocyte donation cycles, during oocyte freezing for fertility preservation and finally, objective oocyte scoring can be important in cases of very poor treatment outcome as a tool for explanation of results to the patient.

Artificial intelligence in the embryology laboratory: a review

The goal of an IVF cycle is a healthy live-born baby. Despite the many advances in the field of assisted reproductive technologies, accurately predicting the outcome of an IVF cycle has yet to be achieved. One reason for this is the method of selecting an embryo for transfer. Morphological assessment of embryos is the traditional method of evaluating embryo quality and selecting which embryo to transfer. However, this subjective method of assessing embryos leads to inter- and intra-observer variability, resulting in less than optimal IVF success rates. To overcome this, it is common practice to transfer more than one embryo, potentially resulting in high-risk multiple pregnancies. Although time-lapse incubators and preimplantation genetic testing for aneuploidy have been introduced to help increase the chances of live birth, the outcomes remain less than ideal. Utilization of artificial intelligence (AI) has become increasingly popular in the medical field and is increasingly being leveraged in the embryology laboratory to help improve IVF outcomes. Many studies have been published investigating the use of AI as an unbiased, automated approach to embryo assessment. This review summarizes recent AI advancements in the embryology laboratory.