Characterization of an artificial intelligence model for ranking static images of blastocyst stage embryos

Impact of inclusion of a poor-quality embryo with a good-quality embryo on pregnancy outcomes in vitrified-warmed blastocyst transfers

RESEARCH QUESTION

Does the co-transfer of a good-quality embryo and a poor-quality embryo influence pregnancy outcomes in comparison to the transfer of a single good-quality embryo in vitrified-warmed blastocyst transfer cycles?

DESIGN

This retrospective cohort study involved a total of 11,738 women who underwent IVF/intracytoplasmic sperm injection cycles and vitrified-warmed blastocyst transfer at a tertiary-care academic medical from January 2015 to June 2022. The study population was categorized into two groups: single-blastocyst transfer (SBT; participants who underwent single good-quality embryo transfer, n = 9338) versus double-blastocyst transfer (DBT; participants who underwent transfers with a poor and a good-quality embryo, n = 2400).

RESULTS

The live birth rate (LBR) was significantly higher in the DBT group in comparison with the SBT group (65.6% versus 56.3%, P < 0.001). Multivariable logistic regression analysis showed that DBT was an independent predictor for LBR with a strong potential impact (adjusted odds ratio 1.55, 95% confidence interval 1.41-1.71; P < 0.001). However, the multiple birth rate was significantly higher in the good-quality embryo and poor-quality embryo group compared with patients undergoing a single good-quality embryo transfer (41.4% versus 1.8%; P < 0.001).

CONCLUSIONS

In vitrified-warmed blastocyst transfer cycles, LBR was higher following DBT with one good-quality and one poor-quality embryo compared with SBT. However, this was at the expense of a marked increase in the likelihood of multiple gestations. Physicians should still balance the benefits and risks of double-embryo transfer.

Making and Selecting the Best Embryo in In vitro Fertilization

Currently, most assisted reproduction units transfer a single embryo to avoid multiple pregnancies. Embryologists must select the embryo to be transferred from a cohort produced by a couple during a cycle. This selection process should be accurate, non-invasive, inexpensive, reproducible, and available to in vitro fertilization (IVF) laboratories worldwide. Embryo selection has evolved from static and morphological criteria to the use of morphokinetic embryonic characteristics using time-lapse systems and artificial intelligence, as well as the genetic study of embryos, both invasive with preimplantation genetic testing for aneuploidies (PGT-A) and non-invasive (niPGT-A). However, despite these advances in embryo selection methods, the overall success rate of IVF techniques remains between 25 and 30%. This review summarizes the different methods and evolution of embryo selection, their strengths and limitations, as well as future technologies that can improve patient outcomes in the shortest possible time. These methodologies are based on procedures that are applied at different stages of embryo development, from the oocyte to the cleavage and blastocyst stages, and can be used in laboratory routine.

Testing an artificial intelligence algorithm to predict fetal heartbeat of vitrified-warmed blastocysts from a single image: predictive ability in different settings

STUDY QUESTION

Could an artificial intelligence (AI) algorithm predict fetal heartbeat from images of vitrified-warmed embryos?

SUMMARY ANSWER

Applying AI to vitrified-warmed blastocysts may help predict which ones will result in implantation failure early enough to thaw another.

WHAT IS KNOWN ALREADY

The application of AI in the field of embryology has already proven effective in assessing the quality of fresh embryos. Therefore, it could also be useful to predict the outcome of frozen embryo transfers, some of which do not recover their pre-vitrification volume, collapse, or degenerate after warming without prior evidence.

STUDY DESIGN, SIZE, DURATION

This retrospective cohort study included 1109 embryos from 792 patients. Of these, 568 were vitrified blastocysts cultured in time-lapse systems in the period between warming and transfer, from February 2022 to July 2023. The other 541 were fresh-transferred blastocysts serving as controls.

PARTICIPANTS/MATERIALS, SETTING, METHODS

Four types of time-lapse images were collected: last frame of development of 541 fresh-transferred blastocysts (FTi), last frame of 467 blastocysts to be vitrified (PVi), first frame post-warming of 568 vitrified embryos (PW1i), and last frame post-warming of 568 vitrified embryos (PW2i). After providing the images to the AI algorithm, the returned scores were compared with the conventional morphology and fetal heartbeat outcomes of the transferred embryos (n = 1098). The contribution of the AI score to fetal heartbeat was analyzed by multivariate logistic regression in different patient populations, and the predictive ability of the models was measured by calculating the area under the receiver-operating characteristic curve (ROC-AUC).

MAIN RESULTS AND THE ROLE OF CHANCE

Fetal heartbeat rate was related to AI score from FTi (P < 0.001), PW1i (P < 0.05), and PW2i (P < 0.001) images. The contribution of AI score to fetal heartbeat was significant in the oocyte donation program for PW2i (odds ratio (OR)=1.13; 95% CI [1.04-1.23]; P < 0.01), and in cycles with autologous oocytes for PW1i (OR = 1.18; 95% CI [1.01-1.38]; P < 0.05) and PW2i (OR = 1.15; 95% CI [1.02-1.30]; P < 0.05), but was not significantly associated with fetal heartbeat in genetically analyzed embryos. AI scores from the four groups of images varied according to morphological category (P < 0.001). The PW2i score differed in collapsed, non-re-expanded, or non-viable embryos compared to normal/viable embryos (P < 0.001). The predictability of the AI score was optimal at a post-warming incubation time of 3.3-4 h (AUC = 0.673).

LIMITATIONS, REASONS FOR CAUTION

The algorithm was designed to assess fresh embryos prior to vitrification, but not thawed ones, so this study should be considered an external trial.

WIDER IMPLICATIONS OF THE FINDINGS

The application of predictive software in the management of frozen embryo transfers may be a useful tool for embryologists, reducing the cancellation rates of cycles in which the blastocyst does not recover from vitrification. Specifically, the algorithm tested in this research could be used to evaluate thawed embryos both in clinics with time-lapse systems and in those with conventional incubators only, as just a single photo is required.

STUDY FUNDING/COMPETING INTERESTS

This study was supported by the Regional Ministry of Innovation, Universities, Science and Digital Society of the Valencian Community (CIACIF/2021/019) and by Instituto de Salud Carlos III (PI21/00283), and co-funded by European Union (ERDF, 'A way to make Europe'). M.M. received personal fees in the last 5 years as honoraria for lectures from Merck, Vitrolife, MSD, Ferring, AIVF, Theramex, Gedeon Richter, Genea Biomedx, and Life Whisperer. There are no other competing interests.

TRIAL REGISTRATION NUMBER

N/A.

Evaluation of the Clinical Efficacy and Trust in AI-Assisted Embryo Ranking: Survey-Based Prospective Study

BACKGROUND

Current embryo assessment methods for in vitro fertilization depend on subjective morphological assessments. Recently, artificial intelligence (AI) has emerged as a promising tool for embryo assessment; however, its clinical efficacy and trustworthiness remain unproven. Simulation studies may provide additional evidence, provided that they are meticulously designed to mitigate bias and variance.

OBJECTIVE

The primary objective of this study was to evaluate the benefits of an AI model for predicting clinical pregnancy through well-designed simulations. The secondary objective was to identify the characteristics of and potential bias in the subgroups of embryologists with varying degrees of experience.

METHODS

This simulation study involved a questionnaire-based survey conducted on 61 embryologists with varying levels of experience from 12 in vitro fertilization clinics. The survey was conducted via Google Forms (Google Inc) in three phases: (1) phase 1, an initial assessment (December 23, 2022, to January 22, 2023); (2) phase 2, a validation assessment (March 6, 2023, to April 5, 2023); and (3) phase 3 an AI-guided assessment (March 6, 2023, to April 5, 2023). Inter- and intraobserver assessments and the accuracy of embryo selection from 360 day-5 embryos before and after AI guidance were analyzed for all embryologists and subgroups of senior and junior embryologists.

RESULTS

With AI guidance, the interobserver agreement increased from 0.355 to 0.527 and from 0.440 to 0.524 for junior and senior embryologists, respectively, thus reaching similar levels of agreement. In a test of accurate embryo selection with 90 questions, the numbers of correct responses by the embryologists only, embryologists with AI guidance, and AI only were 34 (38%), 45 (50%), and 59 (66%), respectively. Without AI, the average score (accuracy) of the junior group was 33.516 (37%), while that of the senior group was 35.967 (40%), with P<.001 in the t test. With AI guidance, the average score (accuracy) of the junior group increased to 46.581 (52%), reaching a level similar to that of the senior embryologists of 44.833 (50%), with P=.34. Junior embryologists had a higher level of trust in the AI score.

CONCLUSIONS

This study demonstrates the potential benefits of AI in selecting embryos with high chances of pregnancy, particularly for embryologists with 5 years or less of experience, possibly due to their trust in AI. Thus, using AI as an auxiliary tool in clinical practice has the potential to improve embryo assessment and increase the probability of a successful pregnancy.

Effect of time post warming to embryo transfer on human blastocyst metabolism and pregnancy outcome

PURPOSE

This study is aiming to test whether variation in post warming culture time impacts blastocyst metabolism or pregnancy outcome.

METHODS

In this single center retrospective cohort study, outcomes of 11,520 single frozen embryo transfer (FET) cycles were analyzed from January 2015 to December 2020. Patient treatments included both natural and programmed cycles. Time categories were determined using the time between blastocyst warming and embryo transfer: 0 (0- <1h), 1 (1-<2h), 2 (2-<3h), 3(3-<4h), 4 (4-<5), 5 (5-<6), 6 (6-<7) and 7 (7-8h). Non-invasive metabolic imaging of discarded human blastocysts for up to 10h was also performed using Fluorescence lifetime imaging microscopy (FLIM) to examine for metabolic perturbations during culture.

RESULTS

The mean age of patients across all time categories were comparable (35.6 ± 3.9). Live birth rates (38-52%) and miscarriage rate (5-11%) were not statistically different across post-warming culture time. When assessing pregnancy outcomes based on the use of PGT-A, miscarriage and live birth rates were not statistically different across culture hours in both PGT-A and non-PGT cycles. Further metabolic analysis of blastocysts for the duration of 10h of culture post warming, revealed minimal metabolic changes of embryos in culture.

CONCLUSION

Overall, our results show that differences in the time of post warming culture have no significant impact on miscarriage or live birth rate for frozen embryo transfers. This information can be beneficial for clinical practices with either minimal staffing or a high number of patient cases.

Improved prediction of clinical pregnancy using artificial intelligence with enhanced inner cell mass and trophectoderm images

This study aimed to assess the performance of an artificial intelligence (AI) model for predicting clinical pregnancy using enhanced inner cell mass (ICM) and trophectoderm (TE) images. In this retrospective study, we included static images of 2555 day-5-blastocysts from seven in vitro fertilization centers in South Korea. The main outcome of the study was the predictive capability of the model to detect clinical pregnancies (gestational sac). Compared with the original embryo images, the use of enhanced ICM and TE images improved the average area under the receiver operating characteristic curve for the AI model from 0.716 to 0.741. Additionally, a gradient-weighted class activation mapping analysis demonstrated that the enhanced image-trained AI model was able to extract features from crucial areas of the embryo in 99% (506/512) of the cases. Particularly, it could extract the ICM and TE. In contrast, the AI model trained on the original images focused on the main areas in only 86% (438/512) of the cases. Our results highlight the potential efficacy of using ICM- and TE-enhanced embryo images when training AI models to predict clinical pregnancy.

A brief history of artificial intelligence embryo selection: from black-box to glass-box

With the exponential growth of computing power and accumulation of embryo image data in recent years, artificial intelligence (AI) is starting to be utilized in embryo selection in IVF. Amongst different AI technologies, machine learning (ML) has the potential to reduce operator-related subjectivity in embryo selection while saving labor time on this task. However, as modern deep learning (DL) techniques, a subcategory of ML, are increasingly used, its integrated black-box attracts growing concern owing to the well-recognized issues regarding lack of interpretability. Currently, there is a lack of randomized controlled trials to confirm the effectiveness of such black-box models. Recently, emerging evidence has shown underperformance of black-box models compared to the more interpretable traditional ML models in embryo selection. Meanwhile, glass-box AI, such as interpretable ML, is being increasingly promoted across a wide range of fields and is supported by its ethical advantages and technical feasibility. In this review, we propose a novel classification system for traditional and AI-driven systems from an embryology standpoint, defining different morphology-based selection approaches with an emphasis on subjectivity, explainability, and interpretability.

Associations between the artificial intelligence scoring system and live birth outcomes in preimplantation genetic testing for aneuploidy cycles

BACKGROUND

Several studies have demonstrated that iDAScore is more accurate in predicting pregnancy outcomes in cycles without preimplantation genetic testing for aneuploidy (PGT-A) compared to KIDScore and the Gardner criteria. However, the effectiveness of iDAScore in cycles with PGT-A has not been thoroughly investigated. Therefore, this study aims to assess the association between artificial intelligence (AI)-based iDAScore (version 1.0) and pregnancy outcomes in single-embryo transfer (SET) cycles with PGT-A.

METHODS

This retrospective study was approved by the Institutional Review Board of Chung Sun Medical University, Taichung, Taiwan. Patients undergoing SET cycles (n = 482) following PGT-A at a single reproductive center between January 2017 and June 2021. The blastocyst morphology and morphokinetics of all embryos were evaluated using a time-lapse system. The blastocysts were ranked based on the scores generated by iDAScore, which were defined as AI scores, or by KIDScore D5 (version 3.2) following the manufacturer's protocols. A single blastocyst without aneuploidy was transferred after examining the embryonic ploidy status using a next-generation sequencing-based PGT-A platform. Logistic regression analysis with generalized estimating equations was conducted to assess whether AI scores are associated with the probability of live birth (LB) while considering confounding factors.

RESULTS

Logistic regression analysis revealed that AI score was significantly associated with LB probability (adjusted odds ratio [OR] = 2.037, 95% confidence interval [CI]: 1.632-2.542) when pulsatility index (PI) level and types of chromosomal abnormalities were controlled. Blastocysts were divided into quartiles in accordance with their AI score (group 1: 3.0-7.8; group 2: 7.9-8.6; group 3: 8.7-8.9; and group 4: 9.0-9.5). Group 1 had a lower LB rate (34.6% vs. 59.8-72.3%) and a higher rate of pregnancy loss (26% vs. 4.7-8.9%) compared with the other groups (p < 0.05). The receiver operating characteristic curve analysis verified that the iDAScore had a significant but limited ability to predict LB (area under the curve [AUC] = 0.64); this ability was significantly weaker than that of the combination of iDAScore, type of chromosomal abnormalities, and PI level (AUC = 0.67). In the comparison of the LB groups with the non-LB groups, the AI scores were significantly lower in the non-LB groups, both for euploid (median: 8.6 vs. 8.8) and mosaic (median: 8.0 vs. 8.6) SETs.

CONCLUSIONS

Although its predictive ability can be further enhanced, the AI score was significantly associated with LB probability in SET cycles. Euploid or mosaic blastocysts with low AI scores (≤ 7.8) were associated with a lower LB rate, indicating the potential of this annotation-free AI system as a decision-support tool for deselecting embryos with poor pregnancy outcomes following PGT-A.

Application of artificial intelligence in gametes and embryos selection

Gamete and embryo quality are critical to the success rate of Assisted Reproductive Technology (ART) cycles, but there remains a lack of methods to accurately measure the quality of sperm, oocytes and embryos. The ability of Artificial Intelligence (AI) technology to analyze large amounts of data, especially video and images, is particularly useful in gamete and embryo assessment and selection. The well-trained model has fast calculation speed and high accuracy, which can help embryologists to perform more objective gamete and embryo selection. Various artificial intelligence models have been developed for gamete and embryo assessment, some of which exhibit good performance. In this review, we summarize the latest applications of AI technology in semen analysis, as well as selection for sperm, oocyte and embryo, and discuss the existing problems and development directions of artificial intelligence in this field.

An artificial intelligence algorithm for automated blastocyst morphometric parameters demonstrates a positive association with implantation potential

Blastocyst selection is primarily based on morphological scoring systems and morphokinetic data. These methods involve subjective grading and time-consuming techniques. Artificial intelligence allows for objective and quick blastocyst selection. In this study, 608 blastocysts were selected for transfer using morphokinetics and Gardner criteria. Retrospectively, morphometric parameters of blastocyst size, inner cell mass (ICM) size, ICM-to-blastocyst size ratio, and ICM shape were automatically measured by a semantic segmentation neural network model. The model was trained on 1506 videos with 102 videos for validation with no overlap between the ICM and trophectoderm models. Univariable logistic analysis found blastocyst size and ICM-to-blastocyst size ratio to be significantly associated with implantation potential. Multivariable regression analysis, adjusted for woman age, found blastocyst size to be significantly associated with implantation potential. The odds of implantation increased by 1.74 for embryos with a blastocyst size greater than the mean (147 ± 19.1 μm). The performance of the algorithm was represented by an area under the curve of 0.70 (p < 0.01). In conclusion, this study supports the association of a large blastocyst size with higher implantation potential and suggests that automatically measured blastocyst morphometrics can be used as a precise, consistent, and time-saving tool for improving blastocyst selection.

Making and selecting the best embryo in the laboratory

Over the past 4 decades our ability to maintain a viable human embryo in vitro has improved dramatically, leading to higher implantation rates. This has led to a notable shift to single blastocyst transfer and the ensuing elimination of high order multiple gestations. Future improvements to embryo culture systems will not only come from new improved innovative media formulations (such as the inclusion of antioxidants), but plausibly by moving away from static culture to more dynamic perfusion-based systems now made a reality owing to the breakthroughs in three-dimensional printing technology and micro fabrication. Such an approach has already made it feasible to create high resolution devices for intracytoplasmic sperm injection, culture, and cryopreservation, paving the way not only for improvements in outcomes but also automation of assisted reproductive technology. Although improvements in culture systems can lead to further increases in pregnancy outcomes, the ability to quantitate biomarkers of embryo health and viability will reduce time to pregnancy and decrease pregnancy loss. Currently artificial intelligence is being used to assess embryo development through image analysis, but we predict its power will be realized through the creation of selection algorithms based on the integration of information related to metabolic functions, cell-free DNA, and morphokinetics, thereby using vast amounts of different data types obtained for each embryo to predict outcomes. All of this will not only make assisted reproductive technology more effective, but it will also make it more cost effective, thereby increasing patient access to infertility treatment worldwide.

Comparing performance between clinics of an embryo evaluation algorithm based on time-lapse images and machine learning

PURPOSE

This article aims to assess how differences in maternal age distributions between IVF clinics affect the performance of an artificial intelligence model for embryo viability prediction and proposes a method to account for such differences.

METHODS

Using retrospectively collected data from 4805 fresh and frozen single blastocyst transfers of embryos incubated for 5 to 6 days, the discriminative performance was assessed based on fetal heartbeat outcomes. The data was collected from 4 clinics, and the discrimination was measured in terms of the area under ROC curves (AUC) for each clinic. To account for the different age distributions between clinics, a method for age-standardizing the AUCs was developed in which the clinic-specific AUCs were standardized using weights for each embryo according to the relative frequency of the maternal age in the relevant clinic compared to the age distribution in a common reference population.

RESULTS

There was substantial variation in the clinic-specific AUCs with estimates ranging from 0.58 to 0.69 before standardization. The age-standardization of the AUCs reduced the between-clinic variance by 16%. Most notably, three of the clinics had quite similar AUCs after standardization, while the last clinic had a markedly lower AUC both with and without standardization.

CONCLUSION

The method of using age-standardization of the AUCs that is proposed in this article mitigates some of the variability between clinics. This enables a comparison of clinic-specific AUCs where the difference in age distributions is accounted for.

Embryo selection through artificial intelligence versus embryologists: a systematic review

STUDY QUESTION

What is the present performance of artificial intelligence (AI) decision support during embryo selection compared to the standard embryo selection by embryologists?

SUMMARY ANSWER

AI consistently outperformed the clinical teams in all the studies focused on embryo morphology and clinical outcome prediction during embryo selection assessment.

WHAT IS KNOWN ALREADY

The ART success rate is ∼30%, with a worrying trend of increasing female age correlating with considerably worse results. As such, there have been ongoing efforts to address this low success rate through the development of new technologies. With the advent of AI, there is potential for machine learning to be applied in such a manner that areas limited by human subjectivity, such as embryo selection, can be enhanced through increased objectivity. Given the potential of AI to improve IVF success rates, it remains crucial to review the performance between AI and embryologists during embryo selection.

STUDY DESIGN SIZE DURATION

The search was done across PubMed, EMBASE, Ovid Medline, and IEEE Xplore from 1 June 2005 up to and including 7 January 2022. Included articles were also restricted to those written in English. Search terms utilized across all databases for the study were: ('Artificial intelligence' OR 'Machine Learning' OR 'Deep learning' OR 'Neural network') AND ('IVF' OR 'in vitro fertili*' OR 'assisted reproductive techn*' OR 'embryo'), where the character '*' refers the search engine to include any auto completion of the search term.

PARTICIPANTS/MATERIALS SETTING METHODS

A literature search was conducted for literature relating to AI applications to IVF. Primary outcomes of interest were accuracy, sensitivity, and specificity of the embryo morphology grade assessments and the likelihood of clinical outcomes, such as clinical pregnancy after IVF treatments. Risk of bias was assessed using the Modified Down and Black Checklist.

MAIN RESULTS AND THE ROLE OF CHANCE

Twenty articles were included in this review. There was no specific embryo assessment day across the studies-Day 1 until Day 5/6 of embryo development was investigated. The types of input for training AI algorithms were images and time-lapse (10/20), clinical information (6/20), and both images and clinical information (4/20). Each AI model demonstrated promise when compared to an embryologist's visual assessment. On average, the models predicted the likelihood of successful clinical pregnancy with greater accuracy than clinical embryologists, signifying greater reliability when compared to human prediction. The AI models performed at a median accuracy of 75.5% (range 59-94%) on predicting embryo morphology grade. The correct prediction (Ground Truth) was defined through the use of embryo images according to post embryologists' assessment following local respective guidelines. Using blind test datasets, the embryologists' accuracy prediction was 65.4% (range 47-75%) with the same ground truth provided by the original local respective assessment. Similarly, AI models had a median accuracy of 77.8% (range 68-90%) in predicting clinical pregnancy through the use of patient clinical treatment information compared to 64% (range 58-76%) when performed by embryologists. When both images/time-lapse and clinical information inputs were combined, the median accuracy by the AI models was higher at 81.5% (range 67-98%), while clinical embryologists had a median accuracy of 51% (range 43-59%).

LIMITATIONS REASONS FOR CAUTION

The findings of this review are based on studies that have not been prospectively evaluated in a clinical setting. Additionally, a fair comparison of all the studies were deemed unfeasible owing to the heterogeneity of the studies, development of the AI models, database employed and the study design and quality.

WIDER IMPLICATIONS OF THE FINDINGS

AI provides considerable promise to the IVF field and embryo selection. However, there needs to be a shift in developers' perception of the clinical outcome from successful implantation towards ongoing pregnancy or live birth. Additionally, existing models focus on locally generated databases and many lack external validation.

STUDY FUNDING/COMPETING INTERESTS

This study was funded by Monash Data Future Institute. All authors have no conflicts of interest to declare.

REGISTRATION NUMBER

CRD42021256333.

An annotated human blastocyst dataset to benchmark deep learning architectures for in vitro fertilization

Medical Assisted Reproduction proved its efficacy to treat the vast majority forms of infertility. One of the key procedures in this treatment is the selection and transfer of the embryo with the highest developmental potential. To assess this potential, clinical embryologists routinely work with static images (morphological assessment) or short video sequences (time-lapse annotation). Recently, Artificial Intelligence models were utilized to support the embryo selection procedure. Even though they have proven their great potential in different in vitro fertilization settings, there is still considerable room for improvement. To support the advancement of algorithms in this research field, we built a dataset consisting of static blastocyst images and additional annotations. As such, Gardner criteria annotations, depicting a morphological blastocyst rating scheme, and collected clinical parameters are provided. The presented dataset is intended to be used to train deep learning models on static morphological images to predict Gardner's criteria and clinical outcomes such as live birth. A benchmark of human expert's performance in annotating Gardner criteria is provided.

Development and validation of deep learning based embryo selection across multiple days of transfer

This work describes the development and validation of a fully automated deep learning model, iDAScore v2.0, for the evaluation of human embryos incubated for 2, 3, and 5 or more days. We trained and evaluated the model on an extensive and diverse dataset including 181,428 embryos from 22 IVF clinics across the world. To discriminate the transferred embryos with known outcome, we show areas under the receiver operating curve ranging from 0.621 to 0.707 depending on the day of transfer. Predictive performance increased over time and showed a strong correlation with morphokinetic parameters. The model's performance is equivalent to the KIDScore D3 model on day 3 embryos while it significantly surpasses the performance of KIDScore D5 v3 on day 5+ embryos. This model provides an analysis of time-lapse sequences without the need for user input, and provides a reliable method for ranking embryos for their likelihood of implantation, at both cleavage and blastocyst stages. This greatly improves embryo grading consistency and saves time compared to traditional embryo evaluation methods.

Transfer of a poor-quality along with a good-quality embryo on in vitro fertilization/intracytoplasmic sperm injection-embryo transfer clinical outcomes: a systematic review and meta-analysis

OBJECTIVE

To investigate the effect on the pregnancy rate of transfer of a good-quality embryo (GQE) and a poor-quality embryo (PQE) in comparison with a single GQE transfer.

DESIGN

Systematic review and meta-analysis.

SETTING

Not applicable.

PATIENT(S)

Infertility patients undergoing in vitro fertilization/intracytoplasmic sperm injection- embryo transfer.

INTERVENTION(S)

Three major electronic databases (PubMed, Embase, and Cochrane Library) for studies those compared single GQE transfer to double embryo transfer of a GQE + PQE were searched. The Newcastle-Ottawa Quality Assessment Scale was used to evaluate the study quality. Random-effect meta-analysis was performed on all data for an overall analysis, followed by a subgroup analysis (fresh cleavage-stage embryos, fresh blastocysts, frozen-thawed blastocysts, and the same assessment criteria for blastocyst quality).

MAIN OUTCOME MEASURE(S)

The primary outcome was clinical pregnancy rate (CPR).

RESULT(S)

A total of 17 studies with 17,612 cycles for GQE transfer and 6,431 cycles for GQE + PQE transfer were included in the meta-analysis. No significant differences were found in CPR (relative risk [RR] = 1.02; 95% confidence interval [CI], 0.91-1.14) and live birth rate (RR = 0.96; 95% CI, 0.87-1.07) between GQE + PQE and GQE transfers. However, the transfer of GQE + PQE increased multiple pregnancy rate (RR = 0.14; 95% CI, 0.09-0.20) and multiple birth rate (RR = 0.08; 95% CI, 0.06-0.12), when compared with the patients undergoing a single GQE transfer. Subgroup analyses by type of embryo for transfer and assessment criteria for embryo quality showed similar trends.

CONCLUSION(S)

Double embryo transfer with GQE + PQE does not result in increased or decreased CPR and live birth rate when compared with a single GQE transfer but leads to a higher multiple pregnancy rate and multiple birth rate.

CLINICAL TRIAL REGISTRATION NUMBER

Prospero CRD42022296681 (https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=296681) registered on January 7, 2022.

Embryologist agreement when assessing blastocyst implantation probability: is data-driven prediction the solution to embryo assessment subjectivity?

STUDY QUESTION

What is the accuracy and agreement of embryologists when assessing the implantation probability of blastocysts using time-lapse imaging (TLI), and can it be improved with a data-driven algorithm?

SUMMARY ANSWER

The overall interobserver agreement of a large panel of embryologists was moderate and prediction accuracy was modest, while the purpose-built artificial intelligence model generally resulted in higher performance metrics.

WHAT IS KNOWN ALREADY

Previous studies have demonstrated significant interobserver variability amongst embryologists when assessing embryo quality. However, data concerning embryologists' ability to predict implantation probability using TLI is still lacking. Emerging technologies based on data-driven tools have shown great promise for improving embryo selection and predicting clinical outcomes.

STUDY DESIGN, SIZE, DURATION

TLI video files of 136 embryos with known implantation data were retrospectively collected from two clinical sites between 2018 and 2019 for the performance assessment of 36 embryologists and comparison with a deep neural network (DNN).

PARTICIPANTS/MATERIALS, SETTING, METHODS

We recruited 39 embryologists from 13 different countries. All participants were blinded to clinical outcomes. A total of 136 TLI videos of embryos that reached the blastocyst stage were used for this experiment. Each embryo's likelihood of successfully implanting was assessed by 36 embryologists, providing implantation probability grades (IPGs) from 1 to 5, where 1 indicates a very low likelihood of implantation and 5 indicates a very high likelihood. Subsequently, three embryologists with over 5 years of experience provided Gardner scores. All 136 blastocysts were categorized into three quality groups based on their Gardner scores. Embryologist predictions were then converted into predictions of implantation (IPG ≥ 3) and no implantation (IPG ≤ 2). Embryologists' performance and agreement were assessed using Fleiss kappa coefficient. A 10-fold cross-validation DNN was developed to provide IPGs for TLI video files. The model's performance was compared to that of the embryologists.

MAIN RESULTS AND THE ROLE OF CHANCE

Logistic regression was employed for the following confounding variables: country of residence, academic level, embryo scoring system, log years of experience and experience using TLI. None were found to have a statistically significant impact on embryologist performance at α = 0.05. The average implantation prediction accuracy for the embryologists was 51.9% for all embryos (N = 136). The average accuracy of the embryologists when assessing top quality and poor quality embryos (according to the Gardner score categorizations) was 57.5% and 57.4%, respectively, and 44.6% for fair quality embryos. Overall interobserver agreement was moderate (κ = 0.56, N = 136). The best agreement was achieved in the poor + top quality group (κ = 0.65, N = 77), while the agreement in the fair quality group was lower (κ = 0.25, N = 59). The DNN showed an overall accuracy rate of 62.5%, with accuracies of 62.2%, 61% and 65.6% for the poor, fair and top quality groups, respectively. The AUC for the DNN was higher than that of the embryologists overall (0.70 DNN vs 0.61 embryologists) as well as in all of the Gardner groups (DNN vs embryologists-Poor: 0.69 vs 0.62; Fair: 0.67 vs 0.53; Top: 0.77 vs 0.54).

LIMITATIONS, REASONS FOR CAUTION

Blastocyst assessment was performed using video files acquired from time-lapse incubators, where each video contained data from a single focal plane. Clinical data regarding the underlying cause of infertility and endometrial thickness before the transfer was not available, yet may explain implantation failure and lower accuracy of IPGs. Implantation was defined as the presence of a gestational sac, whereas the detection of fetal heartbeat is a more robust marker of embryo viability. The raw data were anonymized to the extent that it was not possible to quantify the number of unique patients and cycles included in the study, potentially masking the effect of bias from a limited patient pool. Furthermore, the lack of demographic data makes it difficult to draw conclusions on how representative the dataset was of the wider population. Finally, embryologists were required to assess the implantation potential, not embryo quality. Although this is not the traditional approach to embryo evaluation, morphology/morphokinetics as a means of assessing embryo quality is believed to be strongly correlated with viability and, for some methods, implantation potential.

WIDER IMPLICATIONS OF THE FINDINGS

Embryo selection is a key element in IVF success and continues to be a challenge. Improving the predictive ability could assist in optimizing implantation success rates and other clinical outcomes and could minimize the financial and emotional burden on the patient. This study demonstrates moderate agreement rates between embryologists, likely due to the subjective nature of embryo assessment. In particular, we found that average embryologist accuracy and agreement were significantly lower for fair quality embryos when compared with that for top and poor quality embryos. Using data-driven algorithms as an assistive tool may help IVF professionals increase success rates and promote much needed standardization in the IVF clinic. Our results indicate a need for further research regarding technological advancement in this field.

STUDY FUNDING/COMPETING INTEREST(S)

Embryonics Ltd is an Israel-based company. Funding for the study was partially provided by the Israeli Innovation Authority, grant #74556.

TRIAL REGISTRATION NUMBER

N/A.