Automatic grading of human blastocysts from time-lapse imaging

Machine learning approaches for image classification in developmental biology and clinical embryology

The rapid increase in the amount of available biological data together with increasing computational power and innovative new machine learning algorithms has resulted in great potential for machine learning approaches to revolutionise image analysis in developmental biology and clinical embryology. In this Spotlight, we provide an introduction to machine learning for developmental biologists interested in incorporating machine learning techniques into their research. We give an overview of essential machine learning concepts and models and describe a few recent examples of how these techniques can be used in developmental biology. We also briefly discuss latest advancements in the field and how it might develop in the future.

Machine learning tool for predicting mature oocyte yield and trigger day from start of stimulation: towards personalized treatment

RESEARCH QUESTION

Can machine learning tools predict the number of metaphase II (MII) oocytes and trigger day at the start of the ovarian stimulation cycle?

DESIGN

A multicentre, retrospective study including 56,490 ovarian stimulation cycles (primary dataset) was carried out between 2020 and 2022 for analysis and feature selection. Of these, 13,090 were used to develop machine learning models for trigger day and the number of MII prediction, and another 5103 ovarian stimulation cycles (clinical validation dataset) from 2023 for clinical validation. Machine learning algorithms using deep learning were developed using optimal features from the primary dataset based on correlation.

RESULTS

A tool with two novel progressive machine learning algorithms using deep learning was able to predict the trigger day and number of MII oocytes: mean absolute error 1.60 (95% CI 1.56 to 1.64) and 3.75 (95% CI 3.65 to 3.86), respectively. The R2 value for the algorithm to predict the number of MII in the interquartile (Q3-Q1/P75-P25) range was 0.88; the entire dataset was 0.70 after removing the outliers at the planning phase of the stimulation cycle, which shows high accuracy. The interquartile root mean square error was 1.10 and 0.66 for the trigger day and the number of oocytes algorithm, respectively.

CONCLUSION

The tool using deep learning algorithms has high prediction power for trigger day and number of MII outcomes, and can be retrieved from patients at the start of the ovarian stimulation cycle; however, inclusion of more data and validation from different clinics are needed.

Application of a methodological framework for the development and multicenter validation of reliable artificial intelligence in embryo evaluation

BACKGROUND

Artificial intelligence (AI) models analyzing embryo time-lapse images have been developed to predict the likelihood of pregnancy following in vitro fertilization (IVF). However, limited research exists on methods ensuring AI consistency and reliability in clinical settings during its development and validation process. We present a methodology for developing and validating an AI model across multiple datasets to demonstrate reliable performance in evaluating blastocyst-stage embryos.

METHODS

This multicenter analysis utilizes time-lapse images, pregnancy outcomes, and morphologic annotations from embryos collected at 10 IVF clinics across 9 countries between 2018 and 2022. The four-step methodology for developing and evaluating the AI model include: (I) curating annotated datasets that represent the intended clinical use case; (II) developing and optimizing the AI model; (III) evaluating the AI's performance by assessing its discriminative power and associations with pregnancy probability across variable data; and (IV) ensuring interpretability and explainability by correlating AI scores with relevant morphologic features of embryo quality. Three datasets were used: the training and validation dataset (n = 16,935 embryos), the blind test dataset (n = 1,708 embryos; 3 clinics), and the independent dataset (n = 7,445 embryos; 7 clinics) derived from previously unseen clinic cohorts.

RESULTS

The AI was designed as a deep learning classifier ranking embryos by score according to their likelihood of clinical pregnancy. Higher AI score brackets were associated with increased fetal heartbeat (FH) likelihood across all evaluated datasets, showing a trend of increasing odds ratios (OR). The highest OR was observed in the top G4 bracket (test dataset G4 score ≥ 7.5: OR 3.84; independent dataset G4 score ≥ 7.5: OR 4.01), while the lowest was in the G1 bracket (test dataset G1 score < 4.0: OR 0.40; independent dataset G1 score < 4.0: OR 0.45). AI score brackets G2, G3, and G4 displayed OR values above 1.0 (P < 0.05), indicating linear associations with FH likelihood. Average AI scores were consistently higher for FH-positive than for FH-negative embryos within each age subgroup. Positive correlations were also observed between AI scores and key morphologic parameters used to predict embryo quality.

CONCLUSIONS

Strong AI performance across multiple datasets demonstrates the value of our four-step methodology in developing and validating the AI as a reliable adjunct to embryo evaluation.

Embryologist staffing in assisted reproductive technology laboratories: An international comparative review

Background

Embryologists are crucial in assisted reproductive technology (ART), yet their duties, education, and licensing requirements vary significantly across countries, complicating the determination of optimal staffing levels in ART laboratories. With anticipated advancements such as automation in ART laboratories, this review comprehensively analyzes factors necessary for appropriate future staffing.

Main Findings

A comprehensive literature search was conducted using PubMed to identify relevant articles up to July 2024, employing keywords such as "embryologist," "staffing," and "certification." Articles were evaluated for content related to laboratory operations, and guidelines from five organizations regarding licensing and education were compared.

Results

The review revealed significant international differences in embryologist certification, duties, and staffing recommendations. These disparities, along with the integration of advanced ART technologies and regulatory requirements, significantly impact future staffing needs in ART laboratories.

Conclusion

The definitions of an ART cycle and required staffing levels vary across organizations, influenced by the certification and duties of embryologists in different countries. Adequate embryologist staffing is essential for ensuring laboratory quality control and impacting patient ART outcomes. As new technologies and automation reshape laboratory workflows, collaborative efforts among organizations, countries, and embryologist associations are essential for developing comprehensive educational curricula and determining appropriate staffing levels.

Deep learning versus manual morphology-based embryo selection in IVF: a randomized, double-blind noninferiority trial

To assess the value of deep learning in selecting the optimal embryo for in vitro fertilization, a multicenter, randomized, double-blind, noninferiority parallel-group trial was conducted across 14 in vitro fertilization clinics in Australia and Europe. Women under 42 years of age with at least two early-stage blastocysts on day 5 were randomized to either the control arm, using standard morphological assessment, or the study arm, employing a deep learning algorithm, intelligent Data Analysis Score (iDAScore), for embryo selection. The primary endpoint was a clinical pregnancy rate with a noninferiority margin of 5%. The trial included 1,066 patients (533 in the iDAScore group and 533 in the morphology group). The iDAScore group exhibited a clinical pregnancy rate of 46.5% (248 of 533 patients), compared to 48.2% (257 of 533 patients) in the morphology arm (risk difference -1.7%; 95% confidence interval -7.7, 4.3; P = 0.62). This study was not able to demonstrate noninferiority of deep learning for clinical pregnancy rate when compared to standard morphology and a predefined prioritization scheme. Australian New Zealand Clinical Trials Registry (ANZCTR) registration: 379161 .

Deep learning pipeline reveals key moments in human embryonic development predictive of live birth after in vitro fertilization

Demand for in vitro fertilization (IVF) treatment is growing; however, success rates remain low partly due to difficulty in selecting the best embryo to be transferred. Current manual assessments are subjective and may not take advantage of the most informative moments in embryo development. Here, we apply convolutional neural networks (CNNs) to identify key windows in pre-implantation human development that can be linked to embryo viability and are therefore suitable for the early grading of IVF embryos. We show how machine learning models trained at these developmental time points can be used to refine overall embryo viability assessment. Exploiting the well-known capabilities of transfer learning, we illustrate the performance of CNN models for very limited datasets, paving the way for the use on a clinic-by-clinic basis, catering for local data heterogeneity.

Artificial intelligence-powered assisted ranking of sibling embryos to increase first cycle pregnancy rate

RESEARCH QUESTION

Could EMBRYOLY, an artificial intelligence embryo evaluation tool, assist embryologists to increase first cycle pregnancy rate and reduce cycles to pregnancy for patients?

DESIGN

Data from 11,988 embryos were collected via EMBRYOLY from 2666 egg retrievals (2019-2022) across 11 centres in France, Spain and Morocco using three time-lapse systems (TLS). Data from two independent clinics were also examined. EMBRYOLY's transformer-based model was applied to transferred embryos to evaluate ranking performances against pregnancy and birth outcomes. It was applied to cohorts to rank sibling embryos (including non-transferred) according to their likelihood of clinical pregnancy and to compute the agreement with the embryologist's highest ranked embryo. Its effect on time to pregnancy and first cycle pregnancy rate was evaluated on cohorts with multiple single blastocyst transfers, assuming the embryologist would have considered EMBRYOLY's ranking on the embryos favoured for transfer.

RESULTS

EMBRYOLY's score correlated significantly with clinical pregnancies and live births for cleavage and blastocyst transfers. This held true for clinical pregnancies from blastocyst transfers in two independent clinics. In cases of multiple single embryo transfers, embryologists achieved a 19.8% first cycle pregnancy rate, which could have been improved to 44.1% with the adjunctive use of EMBRYOLY (McNemar's test: P < 0.001). This could have reduced cycles to clinical pregnancy from 2.01 to 1.66 (Wilcoxon test: P < 0.001).

CONCLUSIONS

EMBRYOLY's potential to enhance first cycle pregnancy rates when combined with embryologists' expertise is highlighted. It reduces the number of unsuccessful cycles for patients across TLS and IVF centres.

BlastAssist: a deep learning pipeline to measure interpretable features of human embryos

STUDY QUESTION

Can the BlastAssist deep learning pipeline perform comparably to or outperform human experts and embryologists at measuring interpretable, clinically relevant features of human embryos in IVF?

SUMMARY ANSWER

The BlastAssist pipeline can measure a comprehensive set of interpretable features of human embryos and either outperform or perform comparably to embryologists and human experts in measuring these features.

WHAT IS KNOWN ALREADY

Some studies have applied deep learning and developed 'black-box' algorithms to predict embryo viability directly from microscope images and videos but these lack interpretability and generalizability. Other studies have developed deep learning networks to measure individual features of embryos but fail to conduct careful comparisons to embryologists' performance, which are fundamental to demonstrate the network's effectiveness.

STUDY DESIGN, SIZE, DURATION

We applied the BlastAssist pipeline to 67 043 973 images (32 939 embryos) recorded in the IVF lab from 2012 to 2017 in Tel Aviv Sourasky Medical Center. We first compared the pipeline measurements of individual images/embryos to manual measurements by human experts for sets of features, including: (i) fertilization status (n = 207 embryos), (ii) cell symmetry (n = 109 embryos), (iii) degree of fragmentation (n = 6664 images), and (iv) developmental timing (n = 21 036 images). We then conducted detailed comparisons between pipeline outputs and annotations made by embryologists during routine treatments for features, including: (i) fertilization status (n = 18 922 embryos), (ii) pronuclei (PN) fade time (n = 13 781 embryos), (iii) degree of fragmentation on Day 2 (n = 11 582 embryos), and (iv) time of blastulation (n = 3266 embryos). In addition, we compared the pipeline outputs to the implantation results of 723 single embryo transfer (SET) cycles, and to the live birth results of 3421 embryos transferred in 1801 cycles.

PARTICIPANTS/MATERIALS, SETTING, METHODS

In addition to EmbryoScope™ image data, manual embryo grading and annotations, and electronic health record (EHR) data on treatment outcomes were also included. We integrated the deep learning networks we developed for individual features to construct the BlastAssist pipeline. Pearson's χ2 test was used to evaluate the statistical independence of individual features and implantation success. Bayesian statistics was used to evaluate the association of the probability of an embryo resulting in live birth to BlastAssist inputs.

MAIN RESULTS AND THE ROLE OF CHANCE

The BlastAssist pipeline integrates five deep learning networks and measures comprehensive, interpretable, and quantitative features in clinical IVF. The pipeline performs similarly or better than manual measurements. For fertilization status, the network performs with very good parameters of specificity and sensitivity (area under the receiver operating characteristics (AUROC) 0.84-0.94). For symmetry score, the pipeline performs comparably to the human expert at both 2-cell (r = 0.71 ± 0.06) and 4-cell stages (r = 0.77 ± 0.07). For degree of fragmentation, the pipeline (acc = 69.4%) slightly under-performs compared to human experts (acc = 73.8%). For developmental timing, the pipeline (acc = 90.0%) performs similarly to human experts (acc = 91.4%). There is also strong agreement between pipeline outputs and annotations made by embryologists during routine treatments. For fertilization status, the pipeline and embryologists strongly agree (acc = 79.6%), and there is strong correlation between the two measurements (r = 0.683). For degree of fragmentation, the pipeline and embryologists mostly agree (acc = 55.4%), and there is also strong correlation between the two measurements (r = 0.648). For both PN fade time (r = 0.787) and time of blastulation (r = 0.887), there's strong correlation between the pipeline and embryologists. For SET cycles, 2-cell time (P < 0.01) and 2-cell symmetry (P < 0.03) are significantly correlated with implantation success rate, while other features showed correlations with implantation success without statistical significance. In addition, 2-cell time (P < 5 × 10-11), PN fade time (P < 5 × 10-10), degree of fragmentation on Day 3 (P < 5 × 10-4), and 2-cell symmetry (P < 5 × 10-3) showed statistically significant correlation with the probability of the transferred embryo resulting in live birth.

LIMITATIONS, REASONS FOR CAUTION

We have not tested the BlastAssist pipeline on data from other clinics or other time-lapse microscopy (TLM) systems. The association study we conducted with live birth results do not take into account confounding variables, which will be necessary to construct an embryo selection algorithm. Randomized controlled trials (RCT) will be necessary to determine whether the pipeline can improve success rates in clinical IVF.

WIDER IMPLICATIONS OF THE FINDINGS

BlastAssist provides a comprehensive and holistic means of evaluating human embryos. Instead of using a black-box algorithm, BlastAssist outputs meaningful measurements of embryos that can be interpreted and corroborated by embryologists, which is crucial in clinical decision making. Furthermore, the unprecedentedly large dataset generated by BlastAssist measurements can be used as a powerful resource for further research in human embryology and IVF.

STUDY FUNDING/COMPETING INTEREST(S)

This work was supported by Harvard Quantitative Biology Initiative, the NSF-Simons Center for Mathematical and Statistical Analysis of Biology at Harvard (award number 1764269), the National Institute of Heath (award number R01HD104969), the Perelson Fund, and the Sagol fund for embryos and stem cells as part of the Sagol Network. The authors declare no competing interests.

TRIAL REGISTRATION NUMBER

Not applicable.

Embryo Selection in the Context of In Vitro Fertilization

From the Catholic perspective, in vitro fertilization (IVF) is morally problematic because it artificially separates the procreative and unitive aspects of the conjugal act. Embryo selection (ES) in the context of IVF is an injustice against the resulting embryos because it treats them as commodities and works against their right to life by determining their implantation potential in light of their features. The Church opposes the eugenics mentality underlying ES. Meanwhile, the IVF industry increasingly uses artificial intelligence (AI) for ES. However, doing so could worsen the injustice by deepening the disrespect of human lives under the technocratic paradigm. As such, Catholic bioethicists are encouraged to advocate for the Church's teachings with renewed vigor. In this commentary, we will examine (1) ES in the context of IVF, (2) using AI for ES, (3) the moral implications of using AI for ES, and (4) points for further consideration. Summary: Using AI for Embryo selection in the context of IVF deepens the disrespect of human lives under the technocratic paradigm.

Interpretable artificial intelligence-assisted embryo selection improved single-blastocyst transfer outcomes: a prospective cohort study

RESEARCH QUESTION

What is the pregnancy and neonatal outcomes of an interpretable artificial intelligence (AI) model for embryo selection in a prospective clinical trial?

DESIGN

This single-centre prospective cohort study was carried out from October 2021 to March 2022. A total of 330 eligible patients were assigned to their preferred groups, with 250 patients undergoing a fresh single-blastocyst transfer cycle after the exclusion criteria had been applied. For the AI-assisted group (AAG), embryologists selected the embryos for transfer based on the ranking recommendations provided by an interpretable AI system, while with the manual group, embryologists used the Gardner grading system to make their decisions.

RESULTS

The implantation rate was significantly higher in the AAG than the manual group (80.87% versus 68.15%, P = 0.022). No significant difference was found in terms of monozygotic twin rate, miscarriage rate, live birth rate and ectopic pregnancy rate between the groups. Furthermore, there was no significant difference in terms of neonatal outcomes, including gestational weeks, premature birth rate, birth height, birthweight, sex ratio at birth and newborn malformation rate. The consensus rate between the AI and retrospective analysis by the embryologists was significantly higher for good-quality embryos (i.e. grade 4BB or higher) versus poor-quality embryos (i.e. less than 4BB) (84.71% versus 25%, P < 0.001).

CONCLUSIONS

These prospective trial results suggest that the proposed AI system could effectively help embryologists to improve the implantation rate with single-blastocyst transfer compared with traditional manual evaluation methods.

The Role of Artificial Intelligence and Machine Learning in Assisted Reproductive Technologies

Artificial intelligence (AI) and machine learning, the form most commonly used in medicine, offer powerful tools utilizing the strengths of large data sets and intelligent algorithms. These systems can help to revolutionize delivery of treatments, access to medical care, and improvement of outcomes, particularly in the realm of reproductive medicine. Whether that is more robust oocyte and embryo grading or more accurate follicular measurement, AI will be able to aid clinicians, and most importantly patients, in providing the best possible and individualized care. However, despite all of the potential strengths of AI, algorithms are not immune to bias and are vulnerable to the many socioeconomic and demographic biases that current healthcare systems suffer from. Wrong diagnoses as well is furthering of healthcare discrimination are real possibilities if both the capabilities and limitations of AI are not well understood. Armed with appropriate knowledge of how AI can most appropriately operate within medicine, and specifically reproductive medicine, will enable clinicians to both create and utilize machine learning-based innovations for the furthering of reproductive medicine and ultimately achieving the goal of building of healthy families.

Integrating imaging-based classification and transcriptomics for quality assessment of human oocytes according to their reproductive efficiency

PURPOSE

Utilising non-invasive imaging parameters to assess human oocyte fertilisation, development and implantation; and their influence on transcriptomic profiles.

METHODS

A ranking tool was designed using imaging data from 957 metaphase II stage oocytes retrieved from 102 patients undergoing ART. Hoffman modulation contrast microscopy was conducted with an Olympus IX53 microscope. Images were acquired prior to ICSI and processed using ImageJ for optical density and grey-level co-occurrence matrices texture analysis. Single-cell RNA sequencing of twenty-three mature oocytes classified according to their competence was performed.

RESULT(S)

Overall fertilisation, blastulation and implantation rates were 73.0%, 62.6% and 50.8%, respectively. Three different algorithms were produced using binary logistic regression methods based on "optimal" quartiles, resulting in an accuracy of prediction of 76.6%, 67% and 80.7% for fertilisation, blastulation and implantation. Optical density, gradient, inverse difference moment (homogeneity) and entropy (structural complexity) were the parameters with highest predictive properties. The ranking tool showed high sensitivity (68.9-90.8%) but with limited specificity (26.5-62.5%) for outcome prediction. Furthermore, five differentially expressed genes were identified when comparing "good" versus "poor" competent oocytes.

CONCLUSION(S)

Imaging properties can be used as a tool to assess differences in the ooplasm and predict laboratory and clinical outcomes. Transcriptomic analysis suggested that oocytes with lower competence may have compromised cell cycle either by non-reparable DNA damage or insufficient ooplasmic maturation. Further development of algorithms based on image parameters is encouraged, with an increased balanced cohort and validated prospectively in multicentric studies.

The Internet of Things in assisted reproduction

The Internet of Things (IoT) is a network connecting physical objects with sensors, software and internet connectivity for data exchange. Integrating the IoT with medical devices shows promise in healthcare, particularly in IVF laboratories. By leveraging telecommunications, cybersecurity, data management and intelligent systems, the IoT can enable a data-driven laboratory with automation, improved conditions, personalized treatment and efficient workflows. The integration of 5G technology ensures fast and reliable connectivity for real-time data transmission, while blockchain technology secures patient data. Fog computing reduces latency and enables real-time analytics. Microelectromechanical systems enable wearable IoT and miniaturized monitoring devices for tracking IVF processes. However, challenges such as security risks and network issues must be addressed through cybersecurity measures and networking advancements. Clinical embryologists should maintain their expertise and knowledge for safety and oversight, even with IoT in the IVF laboratory.

Deep learning for embryo evaluation using time-lapse: a systematic review of diagnostic test accuracy

OBJECTIVE

This study aimed to investigate the accuracy of convolutional neural network models in the assessment of embryos using time-lapse monitoring.

DATA SOURCES

A systematic search was conducted in PubMed and Web of Science databases from January 2016 to December 2022. The search strategy was carried out by using key words and MeSH (Medical Subject Headings) terms.

STUDY ELIGIBILITY CRITERIA

Studies were included if they reported the accuracy of convolutional neural network models for embryo evaluation using time-lapse monitoring. The review was registered with PROSPERO (International Prospective Register of Systematic Reviews; identification number CRD42021275916).

METHODS

Two reviewer authors independently screened results using the Covidence systematic review software. The full-text articles were reviewed when studies met the inclusion criteria or in any uncertainty. Nonconsensus was resolved by a third reviewer. Risk of bias and applicability were evaluated using the QUADAS-2 tool and the modified Joanna Briggs Institute or JBI checklist.

RESULTS

Following a systematic search of the literature, 22 studies were identified as eligible for inclusion. All studies were retrospective. A total of 522,516 images of 222,998 embryos were analyzed. Three main outcomes were evaluated: successful in vitro fertilization, blastocyst stage classification, and blastocyst quality. Most studies reported >80% accuracy, and embryologists were outperformed in some. Ten studies had a high risk of bias, mostly because of patient bias.

CONCLUSION

The application of artificial intelligence in time-lapse monitoring has the potential to provide more efficient, accurate, and objective embryo evaluation. Models that examined blastocyst stage classification showed the best predictions. Models that predicted live birth had a low risk of bias, used the largest databases, and had external validation, which heightens their relevance to clinical application. Our systematic review is limited by the high heterogeneity among the included studies. Researchers should share databases and standardize reporting.

Application of artificial intelligence in gametes and embryos selection

Gamete and embryo quality are critical to the success rate of Assisted Reproductive Technology (ART) cycles, but there remains a lack of methods to accurately measure the quality of sperm, oocytes and embryos. The ability of Artificial Intelligence (AI) technology to analyze large amounts of data, especially video and images, is particularly useful in gamete and embryo assessment and selection. The well-trained model has fast calculation speed and high accuracy, which can help embryologists to perform more objective gamete and embryo selection. Various artificial intelligence models have been developed for gamete and embryo assessment, some of which exhibit good performance. In this review, we summarize the latest applications of AI technology in semen analysis, as well as selection for sperm, oocyte and embryo, and discuss the existing problems and development directions of artificial intelligence in this field.

An artificial intelligence algorithm for automated blastocyst morphometric parameters demonstrates a positive association with implantation potential

Blastocyst selection is primarily based on morphological scoring systems and morphokinetic data. These methods involve subjective grading and time-consuming techniques. Artificial intelligence allows for objective and quick blastocyst selection. In this study, 608 blastocysts were selected for transfer using morphokinetics and Gardner criteria. Retrospectively, morphometric parameters of blastocyst size, inner cell mass (ICM) size, ICM-to-blastocyst size ratio, and ICM shape were automatically measured by a semantic segmentation neural network model. The model was trained on 1506 videos with 102 videos for validation with no overlap between the ICM and trophectoderm models. Univariable logistic analysis found blastocyst size and ICM-to-blastocyst size ratio to be significantly associated with implantation potential. Multivariable regression analysis, adjusted for woman age, found blastocyst size to be significantly associated with implantation potential. The odds of implantation increased by 1.74 for embryos with a blastocyst size greater than the mean (147 ± 19.1 μm). The performance of the algorithm was represented by an area under the curve of 0.70 (p < 0.01). In conclusion, this study supports the association of a large blastocyst size with higher implantation potential and suggests that automatically measured blastocyst morphometrics can be used as a precise, consistent, and time-saving tool for improving blastocyst selection.

Using Unlabeled Information of Embryo Siblings from the Same Cohort Cycle to Enhance In Vitro Fertilization Implantation Prediction

High-content time-lapse embryo imaging assessed by machine learning is revolutionizing the field of in vitro fertilization (IVF). However, the vast majority of IVF embryos are not transferred to the uterus, and these masses of embryos with unknown implantation outcomes are ignored in current efforts that aim to predict implantation. Here, whether, and to what extent the information encoded within "sibling" embryos from the same IVF cohort contributes to the performance of machine learning-based implantation prediction is explored. First, it is shown that the implantation outcome is correlated with attributes derived from the cohort siblings. Second, it is demonstrated that this unlabeled data boosts implantation prediction performance. Third, the cohort properties driving embryo prediction, especially those that rescued erroneous predictions, are characterized. The results suggest that predictive models for embryo implantation can benefit from the overlooked, widely available unlabeled data of sibling embryos by reducing the inherent noise of the individual transferred embryo.

Embryo selection through artificial intelligence versus embryologists: a systematic review

STUDY QUESTION

What is the present performance of artificial intelligence (AI) decision support during embryo selection compared to the standard embryo selection by embryologists?

SUMMARY ANSWER

AI consistently outperformed the clinical teams in all the studies focused on embryo morphology and clinical outcome prediction during embryo selection assessment.

WHAT IS KNOWN ALREADY

The ART success rate is ∼30%, with a worrying trend of increasing female age correlating with considerably worse results. As such, there have been ongoing efforts to address this low success rate through the development of new technologies. With the advent of AI, there is potential for machine learning to be applied in such a manner that areas limited by human subjectivity, such as embryo selection, can be enhanced through increased objectivity. Given the potential of AI to improve IVF success rates, it remains crucial to review the performance between AI and embryologists during embryo selection.

STUDY DESIGN SIZE DURATION

The search was done across PubMed, EMBASE, Ovid Medline, and IEEE Xplore from 1 June 2005 up to and including 7 January 2022. Included articles were also restricted to those written in English. Search terms utilized across all databases for the study were: ('Artificial intelligence' OR 'Machine Learning' OR 'Deep learning' OR 'Neural network') AND ('IVF' OR 'in vitro fertili*' OR 'assisted reproductive techn*' OR 'embryo'), where the character '*' refers the search engine to include any auto completion of the search term.

PARTICIPANTS/MATERIALS SETTING METHODS

A literature search was conducted for literature relating to AI applications to IVF. Primary outcomes of interest were accuracy, sensitivity, and specificity of the embryo morphology grade assessments and the likelihood of clinical outcomes, such as clinical pregnancy after IVF treatments. Risk of bias was assessed using the Modified Down and Black Checklist.

MAIN RESULTS AND THE ROLE OF CHANCE

Twenty articles were included in this review. There was no specific embryo assessment day across the studies-Day 1 until Day 5/6 of embryo development was investigated. The types of input for training AI algorithms were images and time-lapse (10/20), clinical information (6/20), and both images and clinical information (4/20). Each AI model demonstrated promise when compared to an embryologist's visual assessment. On average, the models predicted the likelihood of successful clinical pregnancy with greater accuracy than clinical embryologists, signifying greater reliability when compared to human prediction. The AI models performed at a median accuracy of 75.5% (range 59-94%) on predicting embryo morphology grade. The correct prediction (Ground Truth) was defined through the use of embryo images according to post embryologists' assessment following local respective guidelines. Using blind test datasets, the embryologists' accuracy prediction was 65.4% (range 47-75%) with the same ground truth provided by the original local respective assessment. Similarly, AI models had a median accuracy of 77.8% (range 68-90%) in predicting clinical pregnancy through the use of patient clinical treatment information compared to 64% (range 58-76%) when performed by embryologists. When both images/time-lapse and clinical information inputs were combined, the median accuracy by the AI models was higher at 81.5% (range 67-98%), while clinical embryologists had a median accuracy of 51% (range 43-59%).

LIMITATIONS REASONS FOR CAUTION

The findings of this review are based on studies that have not been prospectively evaluated in a clinical setting. Additionally, a fair comparison of all the studies were deemed unfeasible owing to the heterogeneity of the studies, development of the AI models, database employed and the study design and quality.

WIDER IMPLICATIONS OF THE FINDINGS

AI provides considerable promise to the IVF field and embryo selection. However, there needs to be a shift in developers' perception of the clinical outcome from successful implantation towards ongoing pregnancy or live birth. Additionally, existing models focus on locally generated databases and many lack external validation.

STUDY FUNDING/COMPETING INTERESTS

This study was funded by Monash Data Future Institute. All authors have no conflicts of interest to declare.

REGISTRATION NUMBER

CRD42021256333.

Deep learning system for classification of ploidy status using time-lapse videos

OBJECTIVE

To develop a spatiotemporal model for de prediction of euploid and aneuploid embryos using time-lapse videos from 10-115 hours after insemination (hpi).

DESIGN

Retrospective study.

MAIN OUTCOME MEASURES

The research used an end-to-end approach to develop an automated artificial intelligence system capable of extracting features from images and classifying them, considering spatiotemporal dependencies. A convolutional neural network extracted the most relevant features from each video frame. A bidirectional long short-term memory layer received this information and analyzed the temporal dependencies, obtaining a low-dimensional feature vector that characterized each video. A multilayer perceptron classified them into 2 groups, euploid and noneuploid.

RESULTS

The model performance in accuracy fell between 0.6170 and 0.7308. A multi-input model with a gate recurrent unit module performed better than others; the precision (or positive predictive value) is 0.8205 for predicting euploidy. Sensitivity, specificity, F1-Score and accuracy are 0.6957, 0.7813, 0.7042, and 0.7308, respectively.

CONCLUSIONS

This article proposes an artificial intelligence solution for prioritizing euploid embryo transfer. We can highlight the identification of a noninvasive method for chromosomal status diagnosis using a deep learning approach that analyzes raw data provided by time-lapse incubators. This method demonstrated potential automation of the evaluation process, allowing spatial and temporal information to encode.

The importance of cytoplasmic strings during early human embryonic development

Objectives: During human in vitro fertilisation (IVF) treatments, embryologists attempt to select the most viable embryos for embryo transfer (ET). Previously, embryos were evaluated based on light microscopic morphological parameters. However, this is currently accomplished by morphokinetic analysis of time-lapse recordings. This technique provides us the opportunity to observe cytoplasmic strings at the blastocyst stage. The aim of this work was to examine the relationship between the presence of cytoplasmic strings (CS) and the embryo viability in human in vitro fertilised embryos. Study design: Herein, we present an evaluation of the morphokinetic data on the development of embryos obtained during IVF treatments performed at the Division of Assisted Reproduction between December 2020 and March 2021. The dynamics of embryo development, embryo morphology, and morphokinetic scores generated by a time-lapse system were compared between the presence of cytoplasmic strings (CS+) and their absence (CS-) at the blastocyst stage. Results: The development of 208 embryos from 78 patients was examined. Moreover, 81.2% of the embryos had CS in the blastocyst stage; 77% of CS existed in embryos created by conventional IVF, while 86% of CS existed in embryos fertilised by intracytoplasmic sperm injection (ICSI) (p = 0.08). A greater number of CS+ embryos developed into a higher quality blastocyst (52.1% vs. 20.5%, p = 0.02). The morphokinetic score values characterising the development of embryos, such as Known Implantation Data Score (KIDScore) and Intelligent Data Analysis (iDAScore), were higher in CS+ groups (KID: 6.1 ± 2.1 vs. 4.7 ± 2.07; iDA: 8.0 ± 1.9 vs. 6.8 ± 2.3, p < 0.01). The dynamics of the early embryo development were similar between the two groups; however, CS+ embryos reached the blastocyst stage significantly earlier (tB: 103.9 h vs. tB: 107.6 h; p = 0.001). Conclusion: Based on our results, the number of embryos with cytoplasmic strings was higher than that without cytoplasmic strings, and its presence is not related to the fertilisation method. These embryos reached the blastocyst stage earlier, and their morphokinetic (KIDScore and iDAScore) parameters were better. All these results suggest that the presence of CS indicates higher embryo viability. The examination of this feature may help us make decisions about the embryos with higher implantation potential.

[Interest of iDAScore (intelligent Data Analysis Score) for embryo selection in routine IVF laboratory practice: Results of a preliminary study]

INTRODUCTION

Embryo selection is a major challenge in ART, especially since the generalization of single embryo transfer, and its optimization could lead to the improvement of clinical results in IVF. Recently, several Artificial Intelligence (AI) models, based on deep-learning such as iDAScore, have been developed. These models, trained on time-lapse videos of embryos with known implantation data, can predict the probability of pregnancy for a given embryo, allowing automatization and standardization in embryo selection.

MATERIAL AND METHODS

In this study, we have compared the hierarchical categorization of 311 D5 blastocysts of iDAScore v1.0 and the embryologists of our unit. These 311 D5 blastocysts have been classified as top (70.1%), good (Q+: 10.6%) and poor (Q-: 19.3%) quality by embryologists according to Gardner classification. Median iDAScores were [9.9-8.4],]8.4-7.5] and]7.5-2.1] for top, good and poor-quality blastocysts respectively.

RESULTS

We observed a significantly concordant categorization between iDAScore and embryologists for top, good and poor-quality blastocysts (respectively, 89.5, 36.4 and 48.3%, P < 10-4). Moreover, the hierarchical categorization of the three best blastocysts between iDAScore and the embryologists was as follow: 1st rank: 71.9%; 2nd rank: 61.6%; 3rd rank: 56.8% (P=0.07). One hundred and fifty-one blastocysts with known implantation data were analyzed. The iDAScore of blastocysts that implanted was significantly higher than those that did not implant (implantation+: 9.10±0.57; implantation-: 8.70±0.95, P=0.003).

CONCLUSION

This preliminary study shows that iDAScore is able to perform a reproducible, reliable and immediate hierarchical classification of blastocysts. Moreover, this tool can identify the blastocysts with the highest implantation potential. If these results confirmed on a larger scale of embryos and patients, IA could revolutionize IVF laboratories by standardizing embryo hierarchical selection.

Deep Learning-Based Quantitative Blastocyst Assessment

Selecting the single best blastocyst based on morphological appearance for implantation is a crucial part of in vitro fertilization (IVF). Various deep learning and computer vision-based methods have recently been applied for assessing blastocyst quality. However, to the best of our knowledge, most previous works utilize classification networks to give a qualitative evaluation. It would be challenging to rank blastocyst quality with the same qualitative result. Thus, this paper proposes a regression network combined with a soft attention mechanism for quantitatively evaluating blastocyst quality. The network outputs a continuous score to represent blastocyst quality precisely rather than some categories. As to the soft attention mechanism, the attention module in the network outputs an activation map (attention map) localizing the regions of interest (ROI, i.e., inner cell mass (ICM)) of microscopic blastocyst images. The generated activation map guides the entire network to predict ICM quality more accurately. The experimental results demonstrate that the proposed method is superior to traditional classification-based networks. Moreover, the visualized activation map makes the proposed network decision more reliable.

An annotated human blastocyst dataset to benchmark deep learning architectures for in vitro fertilization

Medical Assisted Reproduction proved its efficacy to treat the vast majority forms of infertility. One of the key procedures in this treatment is the selection and transfer of the embryo with the highest developmental potential. To assess this potential, clinical embryologists routinely work with static images (morphological assessment) or short video sequences (time-lapse annotation). Recently, Artificial Intelligence models were utilized to support the embryo selection procedure. Even though they have proven their great potential in different in vitro fertilization settings, there is still considerable room for improvement. To support the advancement of algorithms in this research field, we built a dataset consisting of static blastocyst images and additional annotations. As such, Gardner criteria annotations, depicting a morphological blastocyst rating scheme, and collected clinical parameters are provided. The presented dataset is intended to be used to train deep learning models on static morphological images to predict Gardner's criteria and clinical outcomes such as live birth. A benchmark of human expert's performance in annotating Gardner criteria is provided.

Towards Automation in IVF: Pre-Clinical Validation of a Deep Learning-Based Embryo Grading System during PGT-A Cycles

Preimplantation genetic testing for aneuploidies (PGT-A) is arguably the most effective embryo selection strategy. Nevertheless, it requires greater workload, costs, and expertise. Therefore, a quest towards user-friendly, non-invasive strategies is ongoing. Although insufficient to replace PGT-A, embryo morphological evaluation is significantly associated with embryonic competence, but scarcely reproducible. Recently, artificial intelligence-powered analyses have been proposed to objectify and automate image evaluations. iDAScore v1.0 is a deep-learning model based on a 3D convolutional neural network trained on time-lapse videos from implanted and non-implanted blastocysts. It is a decision support system for ranking blastocysts without manual input. This retrospective, pre-clinical, external validation included 3604 blastocysts and 808 euploid transfers from 1232 cycles. All blastocysts were retrospectively assessed through the iDAScore v1.0; therefore, it did not influence embryologists' decision-making process. iDAScore v1.0 was significantly associated with embryo morphology and competence, although AUCs for euploidy and live-birth prediction were 0.60 and 0.66, respectively, which is rather comparable to embryologists' performance. Nevertheless, iDAScore v1.0 is objective and reproducible, while embryologists' evaluations are not. In a retrospective simulation, iDAScore v1.0 would have ranked euploid blastocysts as top quality in 63% of cases with one or more euploid and aneuploid blastocysts, and it would have questioned embryologists' ranking in 48% of cases with two or more euploid blastocysts and one or more live birth. Therefore, iDAScore v1.0 may objectify embryologists' evaluations, but randomized controlled trials are required to assess its clinical value.

A hybrid artificial intelligence model leverages multi-centric clinical data to improve fetal heart rate pregnancy prediction across time-lapse systems

STUDY QUESTION

Can artificial intelligence (AI) algorithms developed to assist embryologists in evaluating embryo morphokinetics be enriched with multi-centric clinical data to better predict clinical pregnancy outcome?

SUMMARY ANSWER

Training algorithms on multi-centric clinical data significantly increased AUC compared to algorithms that only analyzed the time-lapse system (TLS) videos.

WHAT IS KNOWN ALREADY

Several AI-based algorithms have been developed to predict pregnancy, most of them based only on analysis of the time-lapse recording of embryo development. It remains unclear, however, whether considering numerous clinical features can improve the predictive performances of time-lapse based embryo evaluation.

STUDY DESIGN, SIZE, DURATION

A dataset of 9986 embryos (95.60% known clinical pregnancy outcome, 32.47% frozen transfers) from 5226 patients from 14 European fertility centers (in two countries) recorded with three different TLS was used to train and validate the algorithms. A total of 31 clinical factors were collected. A separate test set (447 videos) was used to compare performances between embryologists and the algorithm.

PARTICIPANTS/MATERIALS, SETTING, METHODS

Clinical pregnancy (defined as a pregnancy leading to a fetal heartbeat) outcome was first predicted using a 3D convolutional neural network that analyzed videos of the embryonic development up to 2 or 3 days of development (33% of the database) or up to 5 or 6 days of development (67% of the database). The output video score was then fed as input alongside clinical features to a gradient boosting algorithm that generated a second score corresponding to the hybrid model. AUC was computed across 7-fold of the validation dataset for both models. These predictions were compared to those of 13 senior embryologists made on the test dataset.

MAIN RESULTS AND THE ROLE OF CHANCE

The average AUC of the hybrid model across all 7-fold was significantly higher than that of the video model (0.727 versus 0.684, respectively, P = 0.015; Wilcoxon test). A SHapley Additive exPlanations (SHAP) analysis of the hybrid model showed that the six first most important features to predict pregnancy were morphokinetics of the embryo (video score), oocyte age, total gonadotrophin dose intake, number of embryos generated, number of oocytes retrieved, and endometrium thickness. The hybrid model was shown to be superior to embryologists with respect to different metrics, including the balanced accuracy (P ≤ 0.003; Wilcoxon test). The likelihood of pregnancy was linearly linked to the hybrid score, with increasing odds ratio (maximum P-value = 0.001), demonstrating the ranking capacity of the model. Training individual hybrid models did not improve predictive performance. A clinic hold-out experiment was conducted and resulted in AUCs ranging between 0.63 and 0.73. Performance of the hybrid model did not vary between TLS or between subgroups of embryos transferred at different days of embryonic development. The hybrid model did fare better for patients older than 35 years (P < 0.001; Mann-Whitney test), and for fresh transfers (P < 0.001; Mann-Whitney test).

LIMITATIONS, REASONS FOR CAUTION

Participant centers were located in two countries, thus limiting the generalization of our conclusion to wider subpopulations of patients. Not all clinical features were available for all embryos, thus limiting the performances of the hybrid model in some instances.

WIDER IMPLICATIONS OF THE FINDINGS

Our study suggests that considering clinical data improves pregnancy predictive performances and that there is no need to retrain algorithms at the clinic level unless they follow strikingly different practices. This study characterizes a versatile AI algorithm with similar performance on different time-lapse microscopes and on embryos transferred at different development stages. It can also help with patients of different ages and protocols used but with varying performances, presumably because the task of predicting fetal heartbeat becomes more or less hard depending on the clinical context. This AI model can be made widely available and can help embryologists in a wide range of clinical scenarios to standardize their practices.

STUDY FUNDING/COMPETING INTEREST(S)

Funding for the study was provided by ImVitro with grant funding received in part from BPIFrance (Bourse French Tech Emergence (DOS0106572/00), Paris Innovation Amorçage (DOS0132841/00), and Aide au Développement DeepTech (DOS0152872/00)). A.B.-C. is a co-owner of, and holds stocks in, ImVitro SAS. A.B.-C. and F.D.M. hold a patent for 'Devices and processes for machine learning prediction of in vitro fertilization' (EP20305914.2). A.D., N.D., M.M.F., and F.D.M. are or have been employees of ImVitro and have been granted stock options. X.P.-V. has been paid as a consultant to ImVitro and has been granted stocks options of ImVitro. L.C.-D. and C.G.-S. have undertaken paid consultancy for ImVitro SAS. The remaining authors have no conflicts to declare.

TRIAL REGISTRATION NUMBER

N/A.

New frontiers in embryo selection

Human infertility is a major global public health issue estimated to affect one out of six couples, while the number of assisted reproduction cycles grows impressively year over year. Efforts to alleviate infertility using advanced technology are gaining traction rapidly as infertility has an enormous impact on couples and the potential to destabilize entire societies if replacement birthrates are not achieved. Artificial intelligence (AI) technologies, leveraged by the highly advanced assisted reproductive technology (ART) industry, are a promising addition to the armamentarium of tools available to combat global infertility. This review provides a background for current methodologies in embryo selection, which is a manual, time-consuming, and poorly reproducible task. AI has the potential to improve this process (among many others) in both the clinician's office and the IVF laboratory. Embryo selection is evolving through digital methodologies into an automated procedure, with superior reliability and reproducibility, that is likely to result in higher pregnancy rates for patients. There is an emerging body of data demonstrating the utility of AI applications in multiple areas in the IVF laboratory. AI platforms have been developed to evaluate individual embryologist performance; to provide quality assurance for culture systems; to correlate embryologist's assessments and AI systems; to predict embryo ploidy, implantation, fetal heartbeat, and live birth outcome; and to replace the current "analogue" system of embryo selection with a digital paradigm. AI capability will distinguish high performing, high profit margin, low-cost, and highly successful IVF clinic business models. We think it will become the standard, "new normal" in IVF labs, as rapidly and thoroughly as vitrification, blastocyst culture, and intracytoplasmic sperm injection replaced their predecessor technologies. At the time of this review, the AI technology to automate embryo evaluation and selection has robustly matured, and therefore, it is the main focus of this review.

Modeling three sources of uncertainty in assisted reproductive technologies with probabilistic graphical models

Embryo selection is a critical step in assisted reproduction: good selection criteria are expected to increase the probability of inducing a pregnancy. Machine learning techniques have been applied for implantation prediction or embryo quality assessment, which embryologists can use to make a decision about embryo selection. However, this is a highly uncertain real-world problem, and current proposals do not model always all the sources of uncertainty. We present a novel probabilistic graphical model that accounts for three different sources of uncertainty, the standard embryo and cycle viability, and a third one that represents any unknown factor that can drive a treatment to a failure in otherwise perfect conditions. We derive a parametric learning method based on the Expectation-Maximization strategy, which accounts for uncertainty issues. We empirically analyze the model within a real database consisting of 604 cycles (3125 embryos) carried out at Hospital Donostia (Spain). Embryologists followed the protocol of the Spanish Association for Reproduction Biology Studies (ASEBIR), based on morphological features, for embryo selection. Our model predictions are correlated with the ASEBIR protocol, which validates our model. The benefits of accounting for the different sources of uncertainty and the importance of the cycle characteristics are shown. Considering only transferred embryos, our model does not further discriminate them as implanted or failed, suggesting that the ASEBIR protocol could be understood as a thorough summary of the available morphological features.

Embryologist agreement when assessing blastocyst implantation probability: is data-driven prediction the solution to embryo assessment subjectivity?

STUDY QUESTION

What is the accuracy and agreement of embryologists when assessing the implantation probability of blastocysts using time-lapse imaging (TLI), and can it be improved with a data-driven algorithm?

SUMMARY ANSWER

The overall interobserver agreement of a large panel of embryologists was moderate and prediction accuracy was modest, while the purpose-built artificial intelligence model generally resulted in higher performance metrics.

WHAT IS KNOWN ALREADY

Previous studies have demonstrated significant interobserver variability amongst embryologists when assessing embryo quality. However, data concerning embryologists' ability to predict implantation probability using TLI is still lacking. Emerging technologies based on data-driven tools have shown great promise for improving embryo selection and predicting clinical outcomes.

STUDY DESIGN, SIZE, DURATION

TLI video files of 136 embryos with known implantation data were retrospectively collected from two clinical sites between 2018 and 2019 for the performance assessment of 36 embryologists and comparison with a deep neural network (DNN).

PARTICIPANTS/MATERIALS, SETTING, METHODS

We recruited 39 embryologists from 13 different countries. All participants were blinded to clinical outcomes. A total of 136 TLI videos of embryos that reached the blastocyst stage were used for this experiment. Each embryo's likelihood of successfully implanting was assessed by 36 embryologists, providing implantation probability grades (IPGs) from 1 to 5, where 1 indicates a very low likelihood of implantation and 5 indicates a very high likelihood. Subsequently, three embryologists with over 5 years of experience provided Gardner scores. All 136 blastocysts were categorized into three quality groups based on their Gardner scores. Embryologist predictions were then converted into predictions of implantation (IPG ≥ 3) and no implantation (IPG ≤ 2). Embryologists' performance and agreement were assessed using Fleiss kappa coefficient. A 10-fold cross-validation DNN was developed to provide IPGs for TLI video files. The model's performance was compared to that of the embryologists.

MAIN RESULTS AND THE ROLE OF CHANCE

Logistic regression was employed for the following confounding variables: country of residence, academic level, embryo scoring system, log years of experience and experience using TLI. None were found to have a statistically significant impact on embryologist performance at α = 0.05. The average implantation prediction accuracy for the embryologists was 51.9% for all embryos (N = 136). The average accuracy of the embryologists when assessing top quality and poor quality embryos (according to the Gardner score categorizations) was 57.5% and 57.4%, respectively, and 44.6% for fair quality embryos. Overall interobserver agreement was moderate (κ = 0.56, N = 136). The best agreement was achieved in the poor + top quality group (κ = 0.65, N = 77), while the agreement in the fair quality group was lower (κ = 0.25, N = 59). The DNN showed an overall accuracy rate of 62.5%, with accuracies of 62.2%, 61% and 65.6% for the poor, fair and top quality groups, respectively. The AUC for the DNN was higher than that of the embryologists overall (0.70 DNN vs 0.61 embryologists) as well as in all of the Gardner groups (DNN vs embryologists-Poor: 0.69 vs 0.62; Fair: 0.67 vs 0.53; Top: 0.77 vs 0.54).

LIMITATIONS, REASONS FOR CAUTION

Blastocyst assessment was performed using video files acquired from time-lapse incubators, where each video contained data from a single focal plane. Clinical data regarding the underlying cause of infertility and endometrial thickness before the transfer was not available, yet may explain implantation failure and lower accuracy of IPGs. Implantation was defined as the presence of a gestational sac, whereas the detection of fetal heartbeat is a more robust marker of embryo viability. The raw data were anonymized to the extent that it was not possible to quantify the number of unique patients and cycles included in the study, potentially masking the effect of bias from a limited patient pool. Furthermore, the lack of demographic data makes it difficult to draw conclusions on how representative the dataset was of the wider population. Finally, embryologists were required to assess the implantation potential, not embryo quality. Although this is not the traditional approach to embryo evaluation, morphology/morphokinetics as a means of assessing embryo quality is believed to be strongly correlated with viability and, for some methods, implantation potential.

WIDER IMPLICATIONS OF THE FINDINGS

Embryo selection is a key element in IVF success and continues to be a challenge. Improving the predictive ability could assist in optimizing implantation success rates and other clinical outcomes and could minimize the financial and emotional burden on the patient. This study demonstrates moderate agreement rates between embryologists, likely due to the subjective nature of embryo assessment. In particular, we found that average embryologist accuracy and agreement were significantly lower for fair quality embryos when compared with that for top and poor quality embryos. Using data-driven algorithms as an assistive tool may help IVF professionals increase success rates and promote much needed standardization in the IVF clinic. Our results indicate a need for further research regarding technological advancement in this field.

STUDY FUNDING/COMPETING INTEREST(S)

Embryonics Ltd is an Israel-based company. Funding for the study was partially provided by the Israeli Innovation Authority, grant #74556.

TRIAL REGISTRATION NUMBER

N/A.

Automatic characterization of human embryos at day 4 post-insemination from time-lapse imaging using supervised contrastive learning and inductive transfer learning techniques

BACKGROUND

Embryo morphology is a predictive marker for implantation success and ultimately live births. Viability evaluation and quality grading are commonly used to select the embryo with the highest implantation potential. However, the traditional method of manual embryo assessment is time-consuming and highly susceptible to inter- and intra-observer variability. Automation of this process results in more objective and accurate predictions.

METHOD

In this paper, we propose a novel methodology based on deep learning to automatically evaluate the morphological appearance of human embryos from time-lapse imaging. A supervised contrastive learning framework is implemented to predict embryo viability at day 4 and day 5, and an inductive transfer approach is applied to classify embryo quality at both times.

RESULTS

Results showed that both methods outperformed conventional approaches and improved state-of-the-art embryology results for an independent test set. The viability result achieved an accuracy of 0.8103 and 0.9330 and the quality results reached values of 0.7500 and 0.8001 for day 4 and day 5, respectively. Furthermore, qualitative results kept consistency with the clinical interpretation.

CONCLUSIONS

The proposed methods are up to date with the artificial intelligence literature and have been proven to be promising. Furthermore, our findings represent a breakthrough in the field of embryology in that they study the possibilities of embryo selection at day 4. Moreover, the grad-CAMs findings are directly in line with embryologists' decisions. Finally, our results demonstrated excellent potential for the inclusion of the models in clinical practice.

Search for morphological indicators that predict implantation by principal component analysis using images of blastocyst

Background

Although the current evaluation of human blastocysts is based on the Gardner criteria, there may be other notable parameters. The purpose of our study was to clarify whether the morphology of blastocysts has notable indicators other than the Gardner criteria.

Methods

To find such indicators, we compared blastocysts that showed elevated human chorionic gonadotropin (hCG) levels after transplantation (hCG-positive group; n = 129) and those that did not (hCG-negative group; n = 105) using principal component analysis of pixel brightness of the images.

Results

The comparison revealed that the hCG-positive group had grainy morphology and the hCG-negative group had non-grainy morphology. Classification of the blastocysts by this indicator did not make a difference in Gardner score. Interestingly, all embryos with ≥20% fragmentation were non-grainy. The visual classification based on this analysis was significantly more accurate than the prediction of implantation using the Gardner score ≥3BB. As graininess can be used in combination with the Gardner score, this indicator will enhance current reproductive technologies.

Reporting on the Value of Artificial Intelligence in Predicting the Optimal Embryo for Transfer: A Systematic Review including Data Synthesis

Artificial intelligence (AI) has been gaining support in the field of in vitro fertilization (IVF). Despite the promising existing data, AI cannot yet claim gold-standard status, which serves as the rationale for this study. This systematic review and data synthesis aims to evaluate and report on the predictive capabilities of AI-based prediction models regarding IVF outcome. The study has been registered in PROSPERO (CRD42021242097). Following a systematic search of the literature in Pubmed/Medline, Embase, and Cochrane Central Library, 18 studies were identified as eligible for inclusion. Regarding live-birth, the Area Under the Curve (AUC) of the Summary Receiver Operating Characteristics (SROC) was 0.905, while the partial AUC (pAUC) was 0.755. The Observed: Expected ratio was 1.12 (95%CI: 0.26–2.37; 95%PI: 0.02–6.54). Regarding clinical pregnancy with fetal heartbeat, the AUC of the SROC was 0.722, while the pAUC was 0.774. The O:E ratio was 0.77 (95%CI: 0.54–1.05; 95%PI: 0.21–1.62). According to this data synthesis, the majority of the AI-based prediction models are successful in accurately predicting the IVF outcome regarding live birth, clinical pregnancy, clinical pregnancy with fetal heartbeat, and ploidy status. This review attempted to compare between AI and human prediction capabilities, and although studies do not allow for a meta-analysis, this systematic review indicates that the AI-based prediction models perform rather similarly to the embryologists’ evaluations. While AI models appear marginally more effective, they still have some way to go before they can claim to significantly surpass the clinical embryologists’ predictive competence.

LC-MS Analysis Revealed the Significantly Different Metabolic Profiles in Spent Culture Media of Human Embryos with Distinct Morphology, Karyotype and Implantation Outcomes

In this study we evaluated possible differences in metabolomic profiles of spent embryo culture media (SECM) of human embryos with distinct morphology, karyotype, and implantation outcomes. A total of 153 samples from embryos of patients undergoing in vitro fertilization (IVF) programs were collected and analyzed by HPLC-MS. Metabolomic profiling and statistical analysis revealed clear clustering of day five SECM from embryos with different morphological classes and karyotype. Profiling of day five SECM from embryos with different implantation outcomes showed 241 significantly changed molecular ions in SECM of successfully implanted embryos. Separate analysis of paired SECM samples on days three and five revealed 46 and 29 molecular signatures respectively, significantly differing in culture media of embryos with a successful outcome. Pathway enrichment analysis suggests certain amino acids, vitamins, and lipid metabolic pathways to be crucial for embryo implantation. Differences between embryos with distinct implantation potential are detectable on the third and fifth day of cultivation that may allow the application of culture medium analysis in different transfer protocols for both fresh and cryopreserved embryos. A combination of traditional morphological criteria with metabolic profiling of SECM may increase implantation rates in assisted reproductive technology programs as well as improve our knowledge of the human embryo metabolism in the early stages of development.

Pseudo contrastive labeling for predicting IVF embryo developmental potential

In vitro fertilization is typically associated with high failure rates per transfer, leading to an acute need for the identification of embryos with high developmental potential. Current methods are tailored to specific times after fertilization, often require expert inspection, and have low predictive power. Automatic methods are challenged by ambiguous labels, clinical heterogeneity, and the inability to utilize multiple developmental points. In this work, we propose a novel method that trains a classifier conditioned on the time since fertilization. This classifier is then integrated over time and its output is used to assign soft labels to pairs of samples. The classifier obtained by training on these soft labels presents a significant improvement in accuracy, even as early as 30 h post-fertilization. By integrating the classification scores, the predictive power is further improved. Our results are superior to previously reported methods, including the commercial KIDScore-D3 system, and a group of eight senior professionals, in classifying multiple groups of favorable embryos into groups defined as less favorable based on implantation outcomes, expert decisions based on developmental trajectories, and/or genetic tests.

Robust and generalizable embryo selection based on artificial intelligence and time-lapse image sequences

Assessing and selecting the most viable embryos for transfer is an essential part of in vitro fertilization (IVF). In recent years, several approaches have been made to improve and automate the procedure using artificial intelligence (AI) and deep learning. Based on images of embryos with known implantation data (KID), AI models have been trained to automatically score embryos related to their chance of achieving a successful implantation. However, as of now, only limited research has been conducted to evaluate how embryo selection models generalize to new clinics and how they perform in subgroup analyses across various conditions. In this paper, we investigate how a deep learning-based embryo selection model using only time-lapse image sequences performs across different patient ages and clinical conditions, and how it correlates with traditional morphokinetic parameters. The model was trained and evaluated based on a large dataset from 18 IVF centers consisting of 115,832 embryos, of which 14,644 embryos were transferred KID embryos. In an independent test set, the AI model sorted KID embryos with an area under the curve (AUC) of a receiver operating characteristic curve of 0.67 and all embryos with an AUC of 0.95. A clinic hold-out test showed that the model generalized to new clinics with an AUC range of 0.60-0.75 for KID embryos. Across different subgroups of age, insemination method, incubation time, and transfer protocol, the AUC ranged between 0.63 and 0.69. Furthermore, model predictions correlated positively with blastocyst grading and negatively with direct cleavages. The fully automated iDAScore v1.0 model was shown to perform at least as good as a state-of-the-art manual embryo selection model. Moreover, full automatization of embryo scoring implies fewer manual evaluations and eliminates biases due to inter- and intraobserver variation.

Embryo Grading With Unreliable Labels Due to Chromosome Abnormalities by Regularized PU Learning With Ranking

We propose a method for human embryo grading with its images. This grading has been achieved by positive-negative classification (i.e., live birth or non-live birth). However, negative (non-live birth) labels collected in clinical practice are unreliable because the visual features of negative images are equal to those of positive (live birth) images if these non-live birth embryos have chromosome abnormalities. For alleviating an adverse effect of these unreliable labels, our method employs Positive-Unlabeled (PU) learning so that live birth and non-live birth are labeled as positive and unlabeled, respectively, where unlabeled samples contain both positive and negative samples. In our method, this PU learning on a deep CNN is improved by a learning-to-rank scheme. While the original learning-to-rank scheme is designed for positive-negative learning, it is extended to PU learning. Furthermore, overfitting in this PU learning is alleviated by regularization with mutual information. Experimental results with 643 time-lapse image sequences demonstrate the effectiveness of our framework in terms of the recognition accuracy and the interpretability. In quantitative comparison, the full version of our proposed method outperforms positive-negative classification in recall and F-measure by a wide margin (0.22 vs. 0.69 in recall and 0.27 vs. 0.42 in F-measure). In qualitative evaluation, visual attentions estimated by our method are interpretable in comparison with morphological assessments in clinical practice.

Predicting Embryo Viability Based on Self-Supervised Alignment of Time-Lapse Videos

With self-supervised learning, both labeled and unlabeled data can be used for representation learning and model pretraining. This is particularly relevant when automating the selection of a patient's fertilized eggs (embryos) during a fertility treatment, in which only the embryos that were transferred to the female uterus may have labels of pregnancy. In this paper, we apply a self-supervised video alignment method known as temporal cycle-consistency (TCC) on 38176 time-lapse videos of developing embryos, of which 14550 were labeled. We show how TCC can be used to extract temporal similarities between embryo videos and use these for predicting pregnancy likelihood. Our temporal similarity method outperforms the time alignment measurement (TAM) with an area under the receiver operating characteristic (AUC) of 0.64 vs. 0.56. Compared to existing embryo evaluation models, it places in between a pure temporal and a spatio-temporal model that both require manual annotations. Furthermore, we use TCC for transfer learning in a semi-supervised fashion and show significant performance improvements compared to standard supervised learning, when only a small subset of the dataset is labeled. Specifically, two variants of transfer learning both achieve an AUC of 0.66 compared to 0.63 for supervised learning when 16% of the dataset is labeled.

Blastocyst Morphology Based on Uniform Time-Point Assessments is Correlated With Mosaic Levels in Embryos

Avoiding aneuploid embryo transfers has been shown to improve pregnancy outcomes in patients with implantation failure and pregnancy loss. This retrospective cohort study aims to analyze the correlation of time-lapse (TL)-based variables and numeric blastocyst morphological scores (TLBMSs) with different mosaic levels. In total, 918 biopsied blastocysts with time-lapse assessments at a uniform time-point were subjected to next-generation sequencing–based preimplantation genetic testing for aneuploidy. In consideration of patient- and cycle-related confounding factors, all redefined blastocyst morphology components of low-grade blastocysts, that is, expansion levels (odds ratio [OR] = 0.388, 95% confidence interval [CI] = 0.217–0.695; OR = 0.328, 95% CI = 0.181–0.596; OR = 0.343, 95% CI = 0.179–0.657), inner cell mass grades (OR = 0.563, 95% CI = 0.333–0.962; OR = 0.35, 95% CI = 0.211–0.58; OR = 0.497, 95% CI = 0.274–0.9), and trophectoderm grades (OR = 0.29, 95% CI = 0.178–0.473; OR = 0.242, 95% CI = 0.143–0.411; OR = 0.3, 95% CI = 0.162–0.554), were less correlated with mosaic levels ≤20%, <50%, and ≤80% as compared with those of top-grade blastocysts (p < 0.05). After converting blastocyst morphology grades into scores, high TLBMSs were associated with greater probabilities of mosaic levels ≤20% (OR = 1.326, 95% CI = 1.187–1.481), <50% (OR = 1.425, 95% CI = 1.262–1.608), and ≤80% (OR = 1.351, 95% CI = 1.186–1.539) (p < 0.001). The prediction abilities of TLBMSs were similar for mosaic levels ≤20% (AUC = 0.604, 95% CI = 0.565–0.642), <50% (AUC = 0.634, 95% CI = 0.598–0.671), and ≤80% (AUC = 0.617, 95% CI = 0.576–0.658). In conclusion, detailed evaluation with TL monitoring at the specific time window reveals that redefined blastocyst morphology components and converted numeric TLBMSs are significantly correlated with all of the threshold levels of mosaicism. However, the performance of TLBMSs to differentiate blastocysts with aberrant ploidy risk remains perfectible.

Time Series Prediction of Viable Embryo and Automatic Grading in IVF using Deep Learning

Background:

The process of In Vitro Fertilization (IVF) involves collecting multiple samples of mature eggs that are fertilized with sperms in the IVF laboratory. They are eventually graded, and the most viable embryo out of all the samples is selected for transfer in the mother’s womb for a healthy pregnancy. Currently, the process of grading and selecting the healthiest embryo is performed by visual morphology, and manual records are maintained by embryologists.

Objectives:

Maintaining manual records makes the process very tedious, time-consuming, and error-prone. The absence of a universal grading leads to high subjectivity and low success rate of pregnancy. To improve the chances of pregnancy, multiple embryos are transferred in the womb elevating the risk of multiple pregnancies. In this paper, we propose a deep learning-based method to perform the automatic grading of the embryos using time series prediction with Long Short Term Memory (LSTM) and Convolutional Neural Network (CNN).

Methods:

CNN extracts the features of the images of embryos, and a sequence of such features is fed to LSTM for time series prediction, which gives the final grade.

Results:

Our model gave an ideal accuracy of 100% on training and validation. A comparison of obtained results is made with those obtained from a GRU model as well as other pre-trained models.

Conclusion:

The automated process is robust and eliminates subjectivity. The days-long hard work can now be replaced with our model, which gives the grading within 8 seconds with a GPU.

Artificial intelligence in the embryology laboratory: a review

The goal of an IVF cycle is a healthy live-born baby. Despite the many advances in the field of assisted reproductive technologies, accurately predicting the outcome of an IVF cycle has yet to be achieved. One reason for this is the method of selecting an embryo for transfer. Morphological assessment of embryos is the traditional method of evaluating embryo quality and selecting which embryo to transfer. However, this subjective method of assessing embryos leads to inter- and intra-observer variability, resulting in less than optimal IVF success rates. To overcome this, it is common practice to transfer more than one embryo, potentially resulting in high-risk multiple pregnancies. Although time-lapse incubators and preimplantation genetic testing for aneuploidy have been introduced to help increase the chances of live birth, the outcomes remain less than ideal. Utilization of artificial intelligence (AI) has become increasingly popular in the medical field and is increasingly being leveraged in the embryology laboratory to help improve IVF outcomes. Many studies have been published investigating the use of AI as an unbiased, automated approach to embryo assessment. This review summarizes recent AI advancements in the embryology laboratory.

Artificial intelligence in the fertility clinic: status, pitfalls and possibilities

In recent years, the amount of data produced in the field of ART has increased exponentially. The diversity of data is large, ranging from videos to tabular data. At the same time, artificial intelligence (AI) is progressively used in medical practice and may become a promising tool to improve success rates with ART. AI models may compensate for the lack of objectivity in several critical procedures in fertility clinics, especially embryo and sperm assessments. Various models have been developed, and even though several of them show promising performance, there are still many challenges to overcome. In this review, we present recent research on AI in the context of ART. We discuss the strengths and weaknesses of the presented methods, especially regarding clinical relevance. We also address the pitfalls hampering successful use of AI in the clinic and discuss future possibilities and important aspects to make AI truly useful for ART.

Pregnancy prediction performance of an annotation-free embryo scoring system on the basis of deep learning after single vitrified-warmed blastocyst transfer: a single-center large cohort retrospective study

OBJECTIVE

To analyze the performance of an annotation-free embryo scoring system on the basis of deep learning for pregnancy prediction after single vitrified blastocyst transfer (SVBT) compared with the performance of other blastocyst grading systems dependent on annotation or morphology scores.

DESIGN

A single-center large cohort retrospective study from an independent validation test.

SETTING

Infertility clinic.

PATIENT(S)

Patients who underwent SVBT cycles (3,018 cycles, mean ± SD patient age 39.3 ± 4.0 years).

INTERVENTION(S)

None.

MAIN OUTCOME MEASURE(S)

The pregnancy prediction performances of each embryo scoring model were compared using the area under curve (AUC) for predicting the fetal heartbeat status for each maternal age group.

RESULT(S)

The AUCs of the <35 years age group (n = 389) for pregnancy prediction were 0.72 for iDAScore, 0.66 for KIDScore, and 0.64 for the Gardner criteria. The AUC of iDAScore was significantly greater than those of the other two models. For the 35-37 years age group (n = 514), the AUCs were 0.68, 0.68, and 0.65 for iDAScore, KIDScore, and the Gardner criteria, respectively, and were not significantly different. The AUCs of the 38-40 years age group (n = 796) were 0.67 for iDAScore, 0.65 for KIDScore, and 0.64 for the Gardner criteria, and there were no significant differences. The AUCs of the 41-42 years age group (n = 636) were 0.66, 0.66, and 0.63 for iDAScore, KIDScore, and the Gardner criteria, respectively, and there were no significant differences among the pregnancy prediction models. For the >42 years age group (n = 389), the AUCs were 0.76 for iDAScore, 0.75 for KIDScore, and 0.75 for the Gardner criteria, and there were no significant differences. Thus, iDAScore AUC was either the highest or equal to the highest AUC for all age groups, although a significant difference was observed only in the youngest age group.

CONCLUSION(S)

Our results showed that objective embryo assessment by a completely automatic and annotation-free model, iDAScore, performed as well as or even better than more traditional embryo assessment or annotation-dependent ranking tools. iDAScore could be an optimal pregnancy prediction model after SVBT, especially in young patients.

Artificial intelligence in human in vitro fertilization and embryology

Embryo evaluation and selection embody the aggregate manifestation of the entire in vitro fertilization (IVF) process. It aims to choose the "best" embryos from the larger cohort of fertilized oocytes, the majority of which will be determined to be not viable either as a result of abnormal development or due to chromosomal imbalances. Indeed, it is generally acknowledged that even after embryo selection based on morphology, time-lapse microscopic photography, or embryo biopsy with preimplantation genetic testing, implantation rates in the human are difficult to predict. Our pursuit of enhancing embryo evaluation and selection, as well as increasing live birth rates, will require the adoption of novel technologies. Recently, several artificial intelligence (AI)-based methods have emerged as objective, standardized, and efficient tools for evaluating human embryos. Moreover, AI-based methods can be implemented for other clinical aspects of IVF, such as assessing patient reproductive potential and individualizing gonadotropin stimulation protocols. As AI has the capability to analyze "big" data, the ultimate goal will be to apply AI tools to the analysis of all embryological, clinical, and genetic data in an effort to provide patient-tailored treatments. In this chapter, we present an overview of existing AI technologies in reproductive medicine and envision their potential future applications in the field.

Precision medicine and artificial intelligence: overview and relevance to reproductive medicine

Traditionally, new treatments have been developed for the population at large. Recently, large-scale genomic sequencing analyses have revealed tremendous genetic diversity between individuals. In diseases driven by genetic events such as cancer, genomic sequencing can unravel all the mutations that drive individual tumors. The ability to capture the genetic makeup of individual patients has led to the concept of precision medicine, a modern, technology-driven form of personalized medicine. Precision medicine matches each individual to the best treatment in a way that is tailored to his or her genetic uniqueness. To further personalize medicine, precision medicine increasingly incorporates and integrates data beyond genomics, such as epigenomics and metabolomics, as well as imaging. Increasingly, the robust use and integration of these modalities in precision medicine require the use of artificial intelligence and machine learning. This modern view of precision medicine, adopted early in certain areas of medicine such as cancer, has started to impact the field of reproductive medicine. Here we review the concepts and history of precision medicine and artificial intelligence, highlight their growing impact on reproductive medicine, and outline some of the challenges and limitations that these new fields have encountered in medicine.

Should we freeze it? Agreement on fate of borderline blastocysts is poor and does not improve with a modified blastocyst grading system

STUDY QUESTION

What is the inter-observer agreement among embryologists for decision to freeze blastocysts of borderline morphology and can it be improved with a modified grading system?

SUMMARY ANSWER

The inter-observer agreement among embryologists deciding whether to freeze blastocysts of marginal morphology was low and was not improved by a modified grading system.

WHAT IS KNOWN ALREADY

While previous research on inter-observer variability on the decision of which embryo to transfer from a cohort of blastocysts is good, the impact of grading variability regarding decision to freeze borderline blastocysts has not been investigated. Agreement for inner cell mass (ICM) and trophectoderm (TE) grade is only fair, factors which contribute to the grade that influences decision to freeze.

STUDY DESIGN, SIZE, DURATION

This was a prospective study involving 18 embryologists working at four different IVF clinics within a single organisation between January 2019 and July 2019.

PARTICIPANTS/MATERIALS, SETTING, METHODS

All embryologists currently practicing blastocyst grading at a multi-site organisation were invited to participate. The survey was comprised of blastocyst images in three planes and asked (i) the likelihood of freezing and (ii) whether the blastocyst would be frozen based on visual assessment. Blastocysts varied by quality and were categorised as either top (n = 20), borderline (n = 60) or non-viable/degenerate quality (n = 20). A total of 1800 freeze decisions were assessed. To assess the impact of grading criteria on inter-observer agreement for decision to freeze, the survey was taken once when the embryologists used the Gardner criteria and again 6 months after transitioning to a modified Gardner criterion with four grades for ICM and TE. The fourth grade was introduced with the aim to promote higher levels of agreement for the clinical usability decision when the blastocyst was of marginal quality.

MAIN RESULTS AND THE ROLE OF CHANCE

The inter-observer agreement for decision to freeze was near perfect (kappa 1.0) for top and non-viable/degenerate quality blastocysts, and this was not affected by the blastocysts grading criteria used (top quality; P = 0.330 and non-viable/degenerate quality; P = 0.18). In contrast, the cohort of borderline blastocysts received a mixed freeze rate (average 52.7%) during the first survey, indicative of blastocysts that showed uncertain viability and promoting significant disagreement for decision to freeze among the embryologists (kappa 0.304). After transitioning to a modified Gardner criteria with an additional grading tier, the average freeze rate increased (64.8%; P < 0.0001); however, the inter-observer agreement for decision to freeze was unchanged (kappa 0.301). Therefore, significant disagreement for decision to freeze among embryologists is an ongoing issue not resolved by the two grading criteria assessed here.

LIMITATIONS, REASONS FOR CAUTION

Blastocyst assessment was performed from time-lapse images in three planes, rather than with a microscope in the laboratory. The inter-observer agreement for decision to freeze may be lower for embryologists working in different clinics with different grading protocols.

WIDER IMPLICATIONS OF THE FINDINGS

The decision to freeze a blastocyst with borderline morphology is a common clinical issue that has the potential to arise for any patient during blastocyst culture. Disagreement for decision to freeze these blastocysts, and therefore clinical usability in frozen embryo transfer cycles, affects consistency in patient care due to a potential impact on cumulative live birth rates, as well as financial, emotional and time costs associated with the frozen embryo transfer cycles. We demonstrate significant disagreement for decision to freeze borderline blastocysts among embryologists using the same grading scheme within a large multisite organisation, a phenomenon which was not improved with a modified grading system. Decision-making around borderline embryos is an area requiring further research, especially as studies continue to demonstrate the reduced but modest live birth rates for low quality blastocysts (Grade C). These results provide support for emerging technology for embryo assessment, such as artificial intelligence.

STUDY FUNDING/COMPETING INTEREST(S)

None declared.

TRIAL REGISTRATION NUMBER

Not applicable.

Development of an artificial intelligence-based assessment model for prediction of embryo viability using static images captured by optical light microscopy during IVF

STUDY QUESTION

Can an artificial intelligence (AI)-based model predict human embryo viability using images captured by optical light microscopy?

SUMMARY ANSWER

We have combined computer vision image processing methods and deep learning techniques to create the non-invasive Life Whisperer AI model for robust prediction of embryo viability, as measured by clinical pregnancy outcome, using single static images of Day 5 blastocysts obtained from standard optical light microscope systems.

WHAT IS KNOWN ALREADY

Embryo selection following IVF is a critical factor in determining the success of ensuing pregnancy. Traditional morphokinetic grading by trained embryologists can be subjective and variable, and other complementary techniques, such as time-lapse imaging, require costly equipment and have not reliably demonstrated predictive ability for the endpoint of clinical pregnancy. AI methods are being investigated as a promising means for improving embryo selection and predicting implantation and pregnancy outcomes.

STUDY DESIGN, SIZE, DURATION

These studies involved analysis of retrospectively collected data including standard optical light microscope images and clinical outcomes of 8886 embryos from 11 different IVF clinics, across three different countries, between 2011 and 2018.

PARTICIPANTS/MATERIALS, SETTING, METHODS

The AI-based model was trained using static two-dimensional optical light microscope images with known clinical pregnancy outcome as measured by fetal heartbeat to provide a confidence score for prediction of pregnancy. Predictive accuracy was determined by evaluating sensitivity, specificity and overall weighted accuracy, and was visualized using histograms of the distributions of predictions. Comparison to embryologists’ predictive accuracy was performed using a binary classification approach and a 5-band ranking comparison.

MAIN RESULTS AND THE ROLE OF CHANCE

The Life Whisperer AI model showed a sensitivity of 70.1% for viable embryos while maintaining a specificity of 60.5% for non-viable embryos across three independent blind test sets from different clinics. The weighted overall accuracy in each blind test set was >63%, with a combined accuracy of 64.3% across both viable and non-viable embryos, demonstrating model robustness and generalizability beyond the result expected from chance. Distributions of predictions showed clear separation of correctly and incorrectly classified embryos. Binary comparison of viable/non-viable embryo classification demonstrated an improvement of 24.7% over embryologists’ accuracy (P = 0.047, n = 2, Student’s t test), and 5-band ranking comparison demonstrated an improvement of 42.0% over embryologists (P = 0.028, n = 2, Student’s t test).

LIMITATIONS, REASONS FOR CAUTION

The AI model developed here is limited to analysis of Day 5 embryos; therefore, further evaluation or modification of the model is needed to incorporate information from different time points. The endpoint described is clinical pregnancy as measured by fetal heartbeat, and this does not indicate the probability of live birth. The current investigation was performed with retrospectively collected data, and hence it will be of importance to collect data prospectively to assess real-world use of the AI model.

WIDER IMPLICATIONS OF THE FINDINGS

These studies demonstrated an improved predictive ability for evaluation of embryo viability when compared with embryologists’ traditional morphokinetic grading methods. The superior accuracy of the Life Whisperer AI model could lead to improved pregnancy success rates in IVF when used in a clinical setting. It could also potentially assist in standardization of embryo selection methods across multiple clinical environments, while eliminating the need for complex time-lapse imaging equipment. Finally, the cloud-based software application used to apply the Life Whisperer AI model in clinical practice makes it broadly applicable and globally scalable to IVF clinics worldwide.

STUDY FUNDING/COMPETING INTEREST(S)

Life Whisperer Diagnostics, Pty Ltd is a wholly owned subsidiary of the parent company, Presagen Pty Ltd. Funding for the study was provided by Presagen with grant funding received from the South Australian Government: Research, Commercialisation and Startup Fund (RCSF). ‘In kind’ support and embryology expertise to guide algorithm development were provided by Ovation Fertility. J.M.M.H., D.P. and M.P. are co-owners of Life Whisperer and Presagen. Presagen has filed a provisional patent for the technology described in this manuscript (52985P pending). A.P.M. owns stock in Life Whisperer, and S.M.D., A.J., T.N. and A.P.M. are employees of Life Whisperer.

Review of computer vision application in in vitro fertilization: the application of deep learning-based computer vision technology in the world of IVF

In vitro fertilization has been regarded as a forefront solution in treating infertility for over four decades, yet its effectiveness has remained relatively low. This could be attributed to the lack of advancements for the method of observing and selecting the most viable embryos for implantation. The conventional morphological assessment of embryos exhibits inevitable drawbacks which include time- and effort-consuming, and imminent risks of bias associated with subjective assessments performed by individual embryologists. A combination of these disadvantages, undeterred by the introduction of the time-lapse incubator technology, has been considered as a prominent contributor to the less preferable success rate of IVF cycles. Nonetheless, a recent surge of AI-based solutions for tasks automation in IVF has been observed. An AI-powered assistant could improve the efficiency of performing certain tasks in addition to offering accurate algorithms that behave as baselines to minimize the subjectivity of the decision-making process. Through a comprehensive review, we have discovered multiple approaches of implementing deep learning technology, each with varying degrees of success, for constructing the automated systems in IVF which could evaluate and even annotate the developmental stages of an embryo.

Development of deep learning algorithms for predicting blastocyst formation and quality by time-lapse monitoring

Approaches to reliably predict the developmental potential of embryos and select suitable embryos for blastocyst culture are needed. The development of time-lapse monitoring (TLM) and artificial intelligence (AI) may help solve this problem. Here, we report deep learning models that can accurately predict blastocyst formation and usable blastocysts using TLM videos of the embryo’s first three days. The DenseNet201 network, focal loss, long short-term memory (LSTM) network and gradient boosting classifier were mainly employed, and video preparation algorithms, spatial stream and temporal stream models were developed into ensemble prediction models called STEM and STEM⁺. STEM exhibited 78.2% accuracy and 0.82 AUC in predicting blastocyst formation, and STEM⁺ achieved 71.9% accuracy and 0.79 AUC in predicting usable blastocysts. We believe the models are beneficial for blastocyst formation prediction and embryo selection in clinical practice, and our modeling methods will provide valuable information for analyzing medical videos with continuous appearance variation.

Liao et al. propose a deep learning model to predict blastocyst formation using TLM videos following the first three days of embryogenesis. The authors develop an ensemble prediction model, STEM and STEM⁺, which were found to exhibit 78.2% and 71.9% accuracy at predicting blastocyst formation and useable blastocysts respectively.

Three-dimensional live imaging of bovine embryos by optical coherence tomography

While embryo transfer (ET) is widely practiced, many of the transferred embryos fail to develop in cattle. To establish a more effective method for selecting bovine embryos for ET, here we quantified morphological parameters of living embryos using three-dimensional (3D) images non-invasively captured by optical coherence tomography (OCT). Seven Japanese Black embryos produced by in vitro fertilization that had reached the expanded blastocyst stage after 7 days of culture were transferred after imaged by OCT. Twenty-two parameters, including thickness and volumes of the inner cell mass, trophectoderm, and zona pellucida, and volumes of blastocoel and whole embryo, were quantified from 3D images. Four of the seven recipients became pregnant. We suggest that these 22 parameters can be potentially employed to evaluate the quality of bovine embryos before ET.

The clinical use of time-lapse in human-assisted reproduction

A major challenge in the assisted reproduction laboratory is to set up reproducible and efficient criteria to identify the embryo with the highest developmental potential. Over the years, several methods have been used worldwide with this purpose. Initially, standard morphology assessment was the only available strategy. It is now universally recognized that besides being a very subjective embryo selection strategy, morphology evaluation alone has a very poor prognostic value. More recently, the availability of time-lapse incubators allowed a continuous monitoring of human embryo development. This technology has spread quickly and many fertility clinics over the world produced a remarkable amount of data. To date, however, a general consensus on which variables, or combination of variables, should play a central role in embryo selection is still lacking. Many confounding factors, concerning both patient features and clinical and biological procedures, have been observed to influence embryo development. In addition, several studies have reported unexpected positive outcomes, even in the presence of abnormal developmental criteria. While it does not seem that time-lapse technology is ready to entirely replace the more invasive preimplantation genetic testing in identifying the embryo with the highest implantation potential, it is certainly true that its application is rapidly growing, becoming progressively more accurate. Studies involving artificial intelligence and deep-learning models as well as combining morphokinetic with other non-invasive markers of embryo development, are currently ongoing, raising hopes for its successful applicability for clinical purpose in the near future. The present review mainly focuses on data published starting from the first decade of 2000, when time-lapse technology was introduced as a routine clinical practice in the infertility centers.