Prospective observational cohort study: Computational models for early prediction of ongoing pregnancy in fresh IVF/ICSI-ET protocols

Comparison of Multivariable Logistic Regression and Other Machine Learning Algorithms for Prognostic Prediction Studies in Pregnancy Care: Systematic Review and Meta-Analysis

BACKGROUND

Predictions in pregnancy care are complex because of interactions among multiple factors. Hence, pregnancy outcomes are not easily predicted by a single predictor using only one algorithm or modeling method.

OBJECTIVE

This study aims to review and compare the predictive performances between logistic regression (LR) and other machine learning algorithms for developing or validating a multivariable prognostic prediction model for pregnancy care to inform clinicians' decision making.

METHODS

Research articles from MEDLINE, Scopus, Web of Science, and Google Scholar were reviewed following several guidelines for a prognostic prediction study, including a risk of bias (ROB) assessment. We report the results based on the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. Studies were primarily framed as PICOTS (population, index, comparator, outcomes, timing, and setting): Population: men or women in procreative management, pregnant women, and fetuses or newborns; Index: multivariable prognostic prediction models using non-LR algorithms for risk classification to inform clinicians' decision making; Comparator: the models applying an LR; Outcomes: pregnancy-related outcomes of procreation or pregnancy outcomes for pregnant women and fetuses or newborns; Timing: pre-, inter-, and peripregnancy periods (predictors), at the pregnancy, delivery, and either puerperal or neonatal period (outcome), and either short- or long-term prognoses (time interval); and Setting: primary care or hospital. The results were synthesized by reporting study characteristics and ROBs and by random effects modeling of the difference of the logit area under the receiver operating characteristic curve of each non-LR model compared with the LR model for the same pregnancy outcomes. We also reported between-study heterogeneity by using τ2 and I2.

RESULTS

Of the 2093 records, we included 142 studies for the systematic review and 62 studies for a meta-analysis. Most prediction models used LR (92/142, 64.8%) and artificial neural networks (20/142, 14.1%) among non-LR algorithms. Only 16.9% (24/142) of studies had a low ROB. A total of 2 non-LR algorithms from low ROB studies significantly outperformed LR. The first algorithm was a random forest for preterm delivery (logit AUROC 2.51, 95% CI 1.49-3.53; I2=86%; τ2=0.77) and pre-eclampsia (logit AUROC 1.2, 95% CI 0.72-1.67; I2=75%; τ2=0.09). The second algorithm was gradient boosting for cesarean section (logit AUROC 2.26, 95% CI 1.39-3.13; I2=75%; τ2=0.43) and gestational diabetes (logit AUROC 1.03, 95% CI 0.69-1.37; I2=83%; τ2=0.07).

CONCLUSIONS

Prediction models with the best performances across studies were not necessarily those that used LR but also used random forest and gradient boosting that also performed well. We recommend a reanalysis of existing LR models for several pregnancy outcomes by comparing them with those algorithms that apply standard guidelines.

TRIAL REGISTRATION

PROSPERO (International Prospective Register of Systematic Reviews) CRD42019136106; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=136106.

Predicting Ectopic Pregnancy Using Human Chorionic Gonadotropin (hCG) Levels and Main Cause of Infertility in Women Undergoing Assisted Reproductive Treatment: Retrospective Observational Cohort Study

Background

Ectopic pregnancy (EP) is a serious complication of assisted reproductive technology (ART). However, there is no acknowledged mathematical model for predicting EP in the ART population.

Objective

The goal of the research was to establish a model to tailor treatment for women with a higher risk of EP.

Methods

From December 2015 to July 2016, we retrospectively included 1703 women whose serum human chorionic gonadotropin (hCG) levels were positive on day 21 (hCG21) after fresh embryo transfer. Multivariable multinomial logistic regression was used to predict EP, intrauterine pregnancy (IUP), and biochemical pregnancy (BCP).

Results

The variables included in the final predicting model were (hCG21, ratio of hCG21/hCG14, and main cause of infertility). During evaluation of the model, the areas under the receiver operating curve for IUP, EP, and BCP were 0.978, 0.962, and 0.999, respectively, in the training set, and 0.963, 0.942, and 0.996, respectively, in the validation set. The misclassification rates were 0.038 and 0.045, respectively, in the training and validation sets. Our model classified the whole in vitro fertilization/intracytoplasmic sperm injection–embryo transfer population into four groups: first, the low-risk EP group, with incidence of EP of 0.52% (0.23%-1.03%); second, a predicted BCP group, with incidence of EP of 5.79% (1.21%-15.95%); third, a predicted undetermined group, with incidence of EP of 28.32% (21.10%-35.53%), and fourth, a predicted high-risk EP group, with incidence of EP of 64.11% (47.22%-78.81%).

Conclusions

We have established a model to sort the women undergoing ART into four groups according to their incidence of EP in order to reduce the medical resources spent on women with low-risk EP and provide targeted tailor-made treatment for women with a higher risk of EP.