Development and validation of prediction models for fetal growth restriction and birthweight: an individual participant data meta-analysis

Background

Fetal growth restriction is associated with perinatal morbidity and mortality. Early identification of women having at-risk fetuses can reduce perinatal adverse outcomes.

Objectives

To assess the predictive performance of existing models predicting fetal growth restriction and birthweight, and if needed, to develop and validate new multivariable models using individual participant data.

Design

Individual participant data meta-analyses of cohorts in International Prediction of Pregnancy Complications network, decision curve analysis and health economics analysis.

Participants

Pregnant women at booking. External validation of existing models (9 cohorts, 441,415 pregnancies); International Prediction of Pregnancy Complications model development and validation (4 cohorts, 237,228 pregnancies).

Predictors

Maternal clinical characteristics, biochemical and ultrasound markers.

Primary outcomes

fetal growth restriction defined as birthweight <10th centile adjusted for gestational age and with stillbirth, neonatal death or delivery before 32 weeks' gestation birthweight.

Analysis

First, we externally validated existing models using individual participant data meta-analysis. If needed, we developed and validated new International Prediction of Pregnancy Complications models using random-intercept regression models with backward elimination for variable selection and undertook internal-external cross-validation. We estimated the study-specific performance (c-statistic, calibration slope, calibration-in-the-large) for each model and pooled using random-effects meta-analysis. Heterogeneity was quantified using τ2 and 95% prediction intervals. We assessed the clinical utility of the fetal growth restriction model using decision curve analysis, and health economics analysis based on National Institute for Health and Care Excellence 2008 model.

Results

Of the 119 published models, one birthweight model (Poon) could be validated. None reported fetal growth restriction using our definition. Across all cohorts, the Poon model had good summary calibration slope of 0.93 (95% confidence interval 0.90 to 0.96) with slight overfitting, and underpredicted birthweight by 90.4 g on average (95% confidence interval 37.9 g to 142.9 g). The newly developed International Prediction of Pregnancy Complications-fetal growth restriction model included maternal age, height, parity, smoking status, ethnicity, and any history of hypertension, pre-eclampsia, previous stillbirth or small for gestational age baby and gestational age at delivery. This allowed predictions conditional on a range of assumed gestational ages at delivery. The pooled apparent c-statistic and calibration were 0.96 (95% confidence interval 0.51 to 1.0), and 0.95 (95% confidence interval 0.67 to 1.23), respectively. The model showed positive net benefit for predicted probability thresholds between 1% and 90%. In addition to the predictors in the International Prediction of Pregnancy Complications-fetal growth restriction model, the International Prediction of Pregnancy Complications-birthweight model included maternal weight, history of diabetes and mode of conception. Average calibration slope across cohorts in the internal-external cross-validation was 1.00 (95% confidence interval 0.78 to 1.23) with no evidence of overfitting. Birthweight was underestimated by 9.7 g on average (95% confidence interval -154.3 g to 173.8 g).

Limitations

We could not externally validate most of the published models due to variations in the definitions of outcomes. Internal-external cross-validation of our International Prediction of Pregnancy Complications-fetal growth restriction model was limited by the paucity of events in the included cohorts. The economic evaluation using the published National Institute for Health and Care Excellence 2008 model may not reflect current practice, and full economic evaluation was not possible due to paucity of data.

Future work

International Prediction of Pregnancy Complications models' performance needs to be assessed in routine practice, and their impact on decision-making and clinical outcomes needs evaluation.

Conclusion

The International Prediction of Pregnancy Complications-fetal growth restriction and International Prediction of Pregnancy Complications-birthweight models accurately predict fetal growth restriction and birthweight for various assumed gestational ages at delivery. These can be used to stratify the risk status at booking, plan monitoring and management.

Study registration

This study is registered as PROSPERO CRD42019135045.

Funding

This award was funded by the National Institute for Health and Care Research (NIHR) Health Technology Assessment programme (NIHR award ref: 17/148/07) and is published in full in Health Technology Assessment; Vol. 28, No. 14. See the NIHR Funding and Awards website for further award information.

Machine Learning Algorithms Versus Classical Regression Models in Pre-Eclampsia Prediction: A Systematic Review

PURPOSE OF REVIEW

Machine learning (ML) approaches are an emerging alternative for healthcare risk prediction. We aimed to synthesise the literature on ML and classical regression studies exploring potential prognostic factors and to compare prediction performance for pre-eclampsia.

RECENT FINDINGS

From 9382 studies retrieved, 82 were included. Sixty-six publications exclusively reported eighty-four classical regression models to predict variable timing of onset of pre-eclampsia. Another six publications reported purely ML algorithms, whilst another 10 publications reported ML algorithms and classical regression models in the same sample with 8 of 10 findings that ML algorithms outperformed classical regression models. The most frequent prognostic factors were age, pre-pregnancy body mass index, chronic medical conditions, parity, prior history of pre-eclampsia, mean arterial pressure, uterine artery pulsatility index, placental growth factor, and pregnancy-associated plasma protein A. Top performing ML algorithms were random forest (area under the curve (AUC) = 0.94, 95% confidence interval (CI) 0.91-0.96) and extreme gradient boosting (AUC = 0.92, 95% CI 0.90-0.94). The competing risk model had similar performance (AUC = 0.92, 95% CI 0.91-0.92) compared with a neural network. Calibration performance was not reported in the majority of publications. ML algorithms had better performance compared to classical regression models in pre-eclampsia prediction. Random forest and boosting-type algorithms had the best prediction performance. Further research should focus on comparing ML algorithms to classical regression models using the same samples and evaluation metrics to gain insight into their performance. External validation of ML algorithms is warranted to gain insights into their generalisability.

Development, validation, and clinical utility of a risk prediction model for recurrent preeclampsia

OBJECTIVES

We aim to establish a predictive model for recurrent preeclampsia.

METHODS

A retrospective review of medical records from three hospitals between 2010 and 2021 was conducted. The study included women who had two consecutive singleton deliveries at the same hospital, with the first delivery complicated by preeclampsia. A multivariable logistic regression model was constructed using a training cohort, and subsequently cross-validated and tested using an independent cohort. The model's performance was assessed in terms of discrimination and calibration, and its clinical utility was evaluated using decision curve analysis (DCA).

RESULTS

Among 296 405 deliveries, 694 women met the inclusion criteria, with 151 (21.8%) experiencing recurrent preeclampsia. The predictive model incorporated 10 risk factors from previous preeclampsia, including gestational weeks with elevated blood pressure, gestational diabetes mellitus (GDM), pericardial effusion, heart failure, limb edema, serum creatinine, white blood cell count, low platelet counts within one week before delivery, SBP on the first postpartum day, and postpartum antihypertensive use. Additionally, one risk factor from the index pregnancy was included, which was antihypertensive use before 20 weeks. The model demonstrated better discrimination, calibration, and a net benefit across a wide range of recurrent preeclampsia risk thresholds. Furthermore, the model has been translated into a clinical risk calculator, enabling clinicians to calculate individualized risks of recurrent preeclampsia.

CONCLUSION

Our study demonstrates that a predictive tool utilizing routine clinical and laboratory factors can accurately estimate the risk of recurrent preeclampsia. This predictive model has the potential to facilitate shared decision-making by providing personalized and risk-stratified care.

Chance of healthy versus adverse outcome in subsequent pregnancy after previous loss beyond 16 weeks: data from a specialized follow-up clinic

PURPOSE

Women with a previous fetal demise have a 2-20 fold increased risk of another stillbirth in a subsequent pregnancy when compared to those who have had a live birth. Despite this, there is limited research regarding the management and outcomes of subsequent pregnancies. This study was conducted to accurately quantify the chances of a woman having a healthy subsequent pregnancy after a pregnancy loss.

METHODS

A retrospective study was conducted at a tertiary-level unit between March 2019 and April 2021. We collected data on all women with a history of previous fetal demise attending a specialized perinatal history clinic and compared the risk of subsequent stillbirth to those with a normal pregnancy outcome. Outcome data included birth outcome, obstetric and medical complications, gestational age and birth weight and mode of delivery. Those who had healthy subsequent pregnancies were compared with those who experienced adverse outcomes.

RESULTS

A total of 101 cases were reviewed. Ninety-six women with subsequent pregnancies after a history of fetal demise from 16 weeks were included. Seventy-nine percent of women (n = 76) delivered a baby at term, without complications. Overall, 2.1% had repeat pregnancy losses (n = 2) and 2.1% delivered babies with fetal growth restriction (n = 2). There were no cases of abruption in a subsequent pregnancy. Eighteen neonates were delivered prematurely (18.4%), 15 of these (83.3%) were due to iatrogenic causes and three (16.7%) were spontaneous. In univariable logistic regression analyses, those with adverse outcomes in subsequent pregnancies had greater odds of pre-eclampsia (Odds ratio *(OR) = 3.89, 95% CI = 1.05-14.43, p = .042) and fetal growth restriction (OR = 4.58, 95% CI = 1.41-14.82, p = 0.011) in previous pregnancies compared to those with healthy outcomes. However, in multivariable logistic regression analyses, neither variable had a significant odds ratio (OR = 2.03, 95% CI = 0.44-9.39, p = .366 and OR = 3.42, 95% CI = 0.90 - 13.09, p = .072 for pre-eclampsia and FGR, respectively).

CONCLUSION

Four in five women had a healthy subsequent pregnancy. This is a reassuring figure for women when contemplating another pregnancy, particularly if cared for in a specialist clinic.

A multistate competing risks framework for preconception prediction of pregnancy outcomes

Background

Preconception pregnancy risk profiles—characterizing the likelihood that a pregnancy attempt results in a full-term birth, preterm birth, clinical pregnancy loss, or failure to conceive—can provide critical information during the early stages of a pregnancy attempt, when obstetricians are best positioned to intervene to improve the chances of successful conception and full-term live birth. Yet the task of constructing and validating risk assessment tools for this earlier intervention window is complicated by several statistical features: the final outcome of the pregnancy attempt is multinomial in nature, and it summarizes the results of two intermediate stages, conception and gestation, whose outcomes are subject to competing risks, measured on different time scales, and governed by different biological processes. In light of this complexity, existing pregnancy risk assessment tools largely focus on predicting a single adverse pregnancy outcome, and make these predictions at some later, post-conception time point.

Methods

We reframe the individual pregnancy attempt as a multistate model comprised of two nested multinomial prediction tasks: one corresponding to conception and the other to the subsequent outcome of that pregnancy. We discuss the estimation of this model in the presence of multiple stages of outcome missingness and then introduce an inverse-probability-weighted Hypervolume Under the Manifold statistic to validate the resulting multivariate risk scores. Finally, we use data from the Effects of Aspirin in Gestation and Reproduction (EAGeR) trial to illustrate how this multistate competing risks framework might be utilized in practice to construct and validate a preconception pregnancy risk assessment tool.

Results

In the EAGeR study population, the resulting risk profiles are able to meaningfully discriminate between the four pregnancy attempt outcomes of interest and represent a significant improvement over classification by random chance.

Conclusions

As illustrated in our analysis of the EAGeR data, our proposed prediction framework expands the pregnancy risk assessment task in two key ways—by considering a broader array of pregnancy outcomes and by providing the predictions at an earlier, preconception intervention window—providing obstetricians and their patients with more information and opportunities to successfully guide pregnancy attempts.

Validation and development of models using clinical, biochemical and ultrasound markers for predicting pre-eclampsia: an individual participant data meta-analysis

BACKGROUND

Pre-eclampsia is a leading cause of maternal and perinatal mortality and morbidity. Early identification of women at risk is needed to plan management.

OBJECTIVES

To assess the performance of existing pre-eclampsia prediction models and to develop and validate models for pre-eclampsia using individual participant data meta-analysis. We also estimated the prognostic value of individual markers.

DESIGN

This was an individual participant data meta-analysis of cohort studies.

SETTING

Source data from secondary and tertiary care.

PREDICTORS

We identified predictors from systematic reviews, and prioritised for importance in an international survey.

PRIMARY OUTCOMES

Early-onset (delivery at < 34 weeks' gestation), late-onset (delivery at ≥ 34 weeks' gestation) and any-onset pre-eclampsia.

ANALYSIS

We externally validated existing prediction models in UK cohorts and reported their performance in terms of discrimination and calibration. We developed and validated 12 new models based on clinical characteristics, clinical characteristics and biochemical markers, and clinical characteristics and ultrasound markers in the first and second trimesters. We summarised the data set-specific performance of each model using a random-effects meta-analysis. Discrimination was considered promising for C-statistics of ≥ 0.7, and calibration was considered good if the slope was near 1 and calibration-in-the-large was near 0. Heterogeneity was quantified using I ² and τ². A decision curve analysis was undertaken to determine the clinical utility (net benefit) of the models. We reported the unadjusted prognostic value of individual predictors for pre-eclampsia as odds ratios with 95% confidence and prediction intervals.

RESULTS

The International Prediction of Pregnancy Complications network comprised 78 studies (3,570,993 singleton pregnancies) identified from systematic reviews of tests to predict pre-eclampsia. Twenty-four of the 131 published prediction models could be validated in 11 UK cohorts. Summary C-statistics were between 0.6 and 0.7 for most models, and calibration was generally poor owing to large between-study heterogeneity, suggesting model overfitting. The clinical utility of the models varied between showing net harm to showing minimal or no net benefit. The average discrimination for IPPIC models ranged between 0.68 and 0.83. This was highest for the second-trimester clinical characteristics and biochemical markers model to predict early-onset pre-eclampsia, and lowest for the first-trimester clinical characteristics models to predict any pre-eclampsia. Calibration performance was heterogeneous across studies. Net benefit was observed for International Prediction of Pregnancy Complications first and second-trimester clinical characteristics and clinical characteristics and biochemical markers models predicting any pre-eclampsia, when validated in singleton nulliparous women managed in the UK NHS. History of hypertension, parity, smoking, mode of conception, placental growth factor and uterine artery pulsatility index had the strongest unadjusted associations with pre-eclampsia.

LIMITATIONS

Variations in study population characteristics, type of predictors reported, too few events in some validation cohorts and the type of measurements contributed to heterogeneity in performance of the International Prediction of Pregnancy Complications models. Some published models were not validated because model predictors were unavailable in the individual participant data.

CONCLUSION

For models that could be validated, predictive performance was generally poor across data sets. Although the International Prediction of Pregnancy Complications models show good predictive performance on average, and in the singleton nulliparous population, heterogeneity in calibration performance is likely across settings.

FUTURE WORK

Recalibration of model parameters within populations may improve calibration performance. Additional strong predictors need to be identified to improve model performance and consistency. Validation, including examination of calibration heterogeneity, is required for the models we could not validate.

STUDY REGISTRATION

This study is registered as PROSPERO CRD42015029349.

FUNDING

This project was funded by the National Institute for Health Research (NIHR) Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 24, No. 72. See the NIHR Journals Library website for further project information.

Angiogenic factors and prediction for ischemic placental disease in future pregnancies

OBJECTIVES

Ischemic placental disease (IPD), including preeclampsia, abruption, and fetal growth restriction, often recurs in subsequent pregnancies. Angiogenic factors of placental origin have been implicated in the pathogenesis of preeclampsia, but have not been studied as predictors of IPD in subsequent pregnancies. We hypothesized that elevated angiogenic factors in an index pregnancy would be associated with recurrence of IPD.

STUDY DESIGN

We conducted a retrospective cohort study of patients undergoing evaluation for preeclampsia who had angiogenic factors measured in an index pregnancy and experienced a subsequent pregnancy at the same institution. Patients with IPD in the index pregnancy were included. A high ratio of soluble fms-like tyrosine kinase 1 (sFlt1) and placental growth factor (PlGF) was defined as greater than or equal to 85.

MAIN OUTCOME MEASURES

The primary outcome was IPD in a subsequent pregnancy.

RESULTS

We included 109 patients in the analysis. The sFlt1/PlGF ratio was elevated in 30% of participants. Those with an elevated ratio were more likely to be nulliparous in the index pregnancy, and less likely to have chronic hypertension. The recurrence of IPD in the study was 27%, with a non-significant difference in risk based on a high sFlt-1/P1GF ratio RR 0.58 (95% CI 0.21 - 1.6) compared to a low ratio.

CONCLUSIONS

A high sFlt1/P1GF ratio in an index pregnancy is not associated with a higher risk of IPD in a subsequent pregnancy. These data suggest placental angiogenic biomarkers are specific to the pregnancy and not a reflection of maternal predisposition to IPD.

Systematic review of prediction models for gestational hypertension and preeclampsia

INTRODUCTION

Prediction models for gestational hypertension and preeclampsia have been developed with data and assumptions from developed countries. Their suitability and application for low resource settings have not been tested. This review aimed to identify and assess the methodological quality of prediction models for gestational hypertension and pre-eclampsia with reference to their application in low resource settings.

METHODS

Using combinations of keywords for gestational hypertension, preeclampsia and prediction models seven databases were searched to identify prediction models developed with maternal data obtained before 20 weeks of pregnancy and including at least three predictors (Prospero registration CRD 42017078786). Prediction model characteristics and performance measures were extracted using the CHARMS, STROBE and TRIPOD checklists. The National Institute of Health quality assessment tools for observational cohort and cross-sectional studies were used for study quality appraisal.

RESULTS

We retrieved 8,309 articles out of which 40 articles were eligible for review. Seventy-seven percent of all the prediction models combined biomarkers with maternal clinical characteristics. Biomarkers used as predictors in most models were pregnancy associated plasma protein-A (PAPP-A) and placental growth factor (PlGF). Only five studies were conducted in a low-and middle income country.

CONCLUSIONS

Most of the studies evaluated did not completely follow the CHARMS, TRIPOD and STROBE guidelines in prediction model development and reporting. Adherence to these guidelines will improve prediction modelling studies and subsequent application of prediction models in clinical practice. Prediction models using maternal characteristics, with good discrimination and calibration, should be externally validated for use in low and middle income countries where biomarker assays are not routinely available.

Prediction models for preeclampsia: A systematic review

BACKGROUND

Preeclampsia is a disease specific to pregnancy that can cause severe maternal and foetal morbidity and mortality. Early identification of women at higher risk for preeclampsia could potentially aid early prevention and treatment. Although a plethora of preeclampsia prediction models have been developed in recent years, individualised prediction of preeclampsia is rarely used in clinical practice.

OBJECTIVES

The objective of this systematic review was to provide an overview of studies on preeclampsia prediction.

STUDY DESIGN

Relevant research papers were identified through a MEDLINE search up to 1 January 2017. Prognostic studies on the prediction of preeclampsia or preeclampsia-related disorders were included. Quality screening was performed with the Quality in Prognostic Studies (QUIPS) tool.

RESULTS

Sixty-eight prediction models from 70 studies with 425,125 participants were selected for further review. The number of participants varied and the gestational age at prediction varied widely across studies. The most frequently used predictors were medical history, body mass index, blood pressure, parity, uterine artery pulsatility index, and maternal age. The type of predictor (maternal characteristics, ultrasound markers and/or biomarkers) was not clearly associated with model discrimination. Few prediction studies were internally (4%) or externally (6%) validated.

CONCLUSIONS

To date, multiple and widely varying models for preeclampsia prediction have been developed, some yielding promising results. The high degree of between-study heterogeneity impedes selection of the best model, or an aggregated analysis of prognostic models. Before multivariable preeclampsia prediction can be clinically implemented universally, further validation and calibration of well-performing prediction models is needed.

Multicenter impact analysis of a model for predicting recurrent early-onset preeclampsia: A before-after study

OBJECTIVE

This study aims to determine the impact of using a prediction model for recurrent preeclampsia to customize antenatal care in subsequent pregnancies.

METHODS

We compared care consumption, pregnancy outcomes, and self-reported health state of two risk-based subgroups, and compared these to a reference group receiving standard care.

RESULTS

We included a total of 311 women from 12 hospitals. Compared to standard care, recurrence-risk guided care did not lead to different outcomes or self-perceived health.

CONCLUSION

Our study exemplifies that recurrence-risk-based stratification of antenatal care in former preeclampsia patients is feasible; it does not lead to worse pregnancy outcomes.

Prognostic models in obstetrics: available, but far from applicable

Health care provision is increasingly focused on the prediction of patients' individual risk for developing a particular health outcome in planning further tests and treatments. There has been a steady increase in the development and publication of prognostic models for various maternal and fetal outcomes in obstetrics. We undertook a systematic review to give an overview of the current status of available prognostic models in obstetrics in the context of their potential advantages and the process of developing and validating models. Important aspects to consider when assessing a prognostic model are discussed and recommendations on how to proceed on this within the obstetric domain are given. We searched MEDLINE (up to July 2012) for articles developing prognostic models in obstetrics. We identified 177 papers that reported the development of 263 prognostic models for 40 different outcomes. The most frequently predicted outcomes were preeclampsia (n = 69), preterm delivery (n = 63), mode of delivery (n = 22), gestational hypertension (n = 11), and small-for-gestational-age infants (n = 10). The performance of newer models was generally not better than that of older models predicting the same outcome. The most important measures of predictive accuracy (ie, a model's discrimination and calibration) were often (82.9%, 218/263) not both assessed. Very few developed models were validated in data other than the development data (8.7%, 23/263). Only two-thirds of the papers (62.4%, 164/263) presented the model such that validation in other populations was possible, and the clinical applicability was discussed in only 11.0% (29/263). The impact of developed models on clinical practice was unknown. We identified a large number of prognostic models in obstetrics, but there is relatively little evidence about their performance, impact, and usefulness in clinical practice so that at this point, clinical implementation cannot be recommended. New efforts should be directed toward evaluating the performance and impact of the existing models.

Care-as-usual provided to formerly preeclamptic women in the Netherlands in the next pregnancy: health care consumption, costs and maternal and child outcome

OBJECTIVE

To explore hospital costs by pregnant women with a history of early-onset preeclampsia or HELLP syndrome, managed according to customary, but non-standardized prenatal care, by relating maternal and child outcome to maternal health care expenditure.

STUDY DESIGN

This was a cohort study, in women of 18 years or older who suffered from early-onset preeclampsia or HELLP syndrome in their previous pregnancy (n=104). We retrieved data retrospectively from hospital information systems and medical records of patients who had received customary, non-standardized prenatal care between 1996 and 2012. Our analyses focused on the costs generated between the first antenatal visit at the outpatient clinic and postpartum hospital discharge. Outcome measures were hospital resource use, costs, maternal and child outcome (recurrence of preeclampsia or HELLP syndrome, incidence of eclampsia, gestational age at delivery, intrauterine fetal demise, small-for-gestational-age birth and low 5min Apgar score). We used linear regression analyses to evaluate whether maternal and child outcome and baseline characteristics correlated with hospital costs.

RESULTS

Maternal hospital costs per patient averaged € 8047. The main cost drivers were maternal admissions and outpatient visits, together accounting for 80% of total costs. Primary cost drivers were preterm birth and recurrent preeclampsia or HELLP syndrome.

CONCLUSION

Hospital costs in the next pregnancy of formerly preeclamptic women varied widely with over 70% being medically unexplainable. The results of this study support the view that care standardization in these women can be expected to improve costs and efficacy of care without compromising outcome.

Early pregnancy circulatory adaptation and recurrent hypertensive disease: an explorative study

INTRODUCTION

Hypertensive pregnancy disorders are assumed to be preceded by defective spiral artery remodeling. Whether this localized aberration at the implantation site affects the initial maternal systemic cardiovascular and renal adaptation to pregnancy is unclear. We explored in a high-risk population, whether the initial systemic maternal adaptation to pregnancy differs between women who do and do not develop a recurrent hypertensive disorder later on in pregnancy.

METHODS

We enrolled 61 normotensive women with a previous hypertensive disorder of pregnancy and subdivided them into 2 subgroups, based on whether or not their next pregnancy remained uneventful (n = 33) or became complicated by a recurrent hypertensive disorder (n = 28). We measured before pregnancy and again at 18 ± 2 weeks of gestation cardiac output, blood pressure, plasma volume, creatinine clearance, and calculated total peripheral vascular resistance from cardiac output and blood pressure.

RESULT

Both subgroups responded to pregnancy with an increase in cardiac output, plasma volume, heart rate, and creatinine clearance, and a decrease in blood pressure and total peripheral vascular resistance. Women who developed a recurrent hypertensive disorder differed from their counterparts with an uneventful next pregnancy by smaller pregnancy-induced increases in creatinine clearance (19% vs. 31%, P = .035) and cardiac output (10% vs. 20%, P = .035), respectively.

CONCLUSION

The initial systemic cardiovascular and renal adaptations to pregnancy in women who develop a recurrent gestational hypertensive disorder differ from those in their counterparts with an uneventful next pregnancy by smaller rises in creatinine clearance and cardiac output.