External validation of models to predict the outcome of pregnancies of unknown location: a multicentre cohort study

Diagnostic accuracy validation study of the M6 model without initial serum progesterone (M6NP) in triage of pregnancy of unknown location

OBJECTIVES

The M6 prediction model stratifies the risk of development of ectopic pregnancy (EP) for women with pregnancy of unknown location (PUL) into low risk or high risk, using human chorionic gonadotrophin (hCG) and progesterone levels at the initial visit to a gynaecological emergency room and hCG level at 48 h. This study evaluated a second model, the M6NP model, which does not include the progesterone level at the initial visit. The main aim of this study was to validate the diagnostic accuracy of the M6NP model in a population of French women.

STUDY DESIGN

Between January and December 2021, all women with an hCG measurement from the gynaecological emergency department of a teaching hospital were screened for inclusion in this study. Women with a pregnancy location determined before or at the second visit were excluded. The diagnostic test was based on logistic regression of the M6NP model, with classification into two groups: high risk of EP (≥5%) and low risk of EP (<5%). The reference test was the final outcome based on clinical, biological and sonographic results: failed PUL (FPUL), intrauterine pregnancy (IUP) or EP. Diagnostic performance for risk prediction of EP, and also IUP and FPUL, was calculated.

RESULTS

In total, 759 women with possible PUL were identified. After screening, 341 women with PUL were included in the main analysis. Of these, 186 (54.5%) were classified as low risk, including three (1.6%) with a final outcome of EP. The remaining 155 women with PUL were classified as high risk, of whom 60 (38.7%), 66 (42.8%) and 29 (18.7%) had a final outcome of FPUL, IUP and EP, respectively. Of the 32 women with PUL with a final outcome of EP, 29 (90.6%) were classified as high risk and three (9.4%) were classified as low risk. Therefore, the performance of the M6NP model to predict EP had a negative predictive value of 98.4%, a positive predictive value of 18.7%, sensitivity of 90.6% and specificity of 59.2%. If the prediction model had been used, it is estimated that 4.5 visits per patient could have been prevented.

CONCLUSION

The M6NP model could be used safely in the French population for risk stratification of PUL. Its use in clinical practice should result in a substantial reduction in the number of visits to a gynaecological emergency room.

Ambulatory human chorionic gonadotrophin (hCG) testing: a verification of two hCG point of care devices

OBJECTIVES

Quantitative human chorionic gonadotropin (hCG) measurements are used to manage women classified with a pregnancy of unknown location (PUL). Two point of care testing (POCT) devices that quantify hCG are commercially available. We verified the i-STAT 1 (Abbott) and the AQT 90 FLEX (Radiometer) prior to use in PUL triage.

METHODS

Tests for precision, external quality assurance (EQA), correlation, hook effect and recovery were undertaken alongside a POCT usability assessment during this prospective multi-center verification.

RESULTS

Coefficients of variation ranged between 4.0 and 5.1 % for the three i-STAT 1 internal quality control (IQC) solutions and between 6.8 and 7.3 % for the two AQT IQC solutions. Symmetric differences in POCT EQA results when compared with laboratory and EQA stock values ranged between 3.2 and 24.5 % for the i-STAT 1 and between 3.3 and 36.9 % for the AQT. Correlation coefficients (i-STAT 1: 0.96, AQT: 0.99) and goodness of fit curves (i-STAT 1: 0.92, AQT: 0.99) were excellent when using suitable whole blood samples. An hCG hook effect was noted with the i-STAT 1 between 572,194 and 799,089 IU/L, lower than the hook effect noted with the AQT, which was between 799,089 and 1,619,309 IU/L. When hematocrit concentration was considered in sample types validated for use with each device, hCG recovery was 108 % with the i-STAT 1 and 98 % with the AQT. The i-STAT 1 scored lower on usability overall (90/130) than the AQT (121/130, p<0.001, Mann-Whitney).

CONCLUSIONS

Both hCG POCT devices were verified for use in clinical practice. Practical factors must also be considered when choosing which device to use in each unit.

Evaluation of novel biomarkers for early pregnancy outcome prediction†

OBJECTIVE

To assess performance and discriminatory capacity of commercially available enzyme-linked immunosorbent assays of biomarkers for predicting first trimester pregnancy outcome in a multi-center cohort.

DESIGN

In a case-control study at three academic centers of women with pain and bleeding in early pregnancy, enzyme-linked immunosorbent assays of biomarkers were screened for assay performance. Performance was assessed via functional sensitivity, assay reportable range, recovery/linearity, and intra-assay precision (%Coefficient of Variation). Top candidates were analyzed for discriminatory capacity for viability and location among 210 women with tubal ectopic pregnancy, viable intrauterine pregnancy, or miscarriage. Assay discrimination was assessed by visual plots, area under the curve with 95% confidence intervals, and measures of central tendency with two-sample t-tests.

RESULTS

Of 25 biomarkers evaluated, 22 demonstrated good or acceptable assay performance. Transgelin-2, oviductal glycoprotein, and integrin-linked kinase were rejected due to poor performance. The best biomarkers for discrimination of pregnancy location were pregnancy-specific beta-1-glycoprotein 9, pregnancy-specific beta-1-glycoprotein 1, insulin-like growth factor binding protein 1, kisspeptin (KISS1), pregnancy-specific beta-1-glycoprotein 3, and beta parvin (PARVB). The best biomarkers for discrimination of pregnancy viability were pregnancy-specific beta-1-glycoprotein 9, pregnancy-specific beta-1-glycoprotein 3, EH domain-containing protein 3, KISS1, WAP four-disulfide core domain protein 2 (HE4), quiescin sulfhydryl oxidase 2, and pregnancy-specific beta-1-glycoprotein 1.

CONCLUSION

Performance of commercially available enzyme-linked immunosorbent assays was acceptable for a panel of novel biomarkers to predict early pregnancy outcome. Of these, six and seven candidates demonstrated good discriminatory capacity of pregnancy location and viability, respectively, when validated in a distinct external population. Four markers demonstrated good discrimination for both location and viability.

Updating M6 pregnancy of unknown location risk-prediction model including evaluation of clinical factors

OBJECTIVES

Ectopic pregnancy (EP) is a major high-risk outcome following a pregnancy of unknown location (PUL) classification. Biochemical markers are used to triage PUL as high vs low risk to guide appropriate follow-up. The M6 model is currently the best risk-prediction model. We aimed to update the M6 model and evaluate whether performance can be improved by including clinical factors.

METHODS

This prospective cohort study recruited consecutive PUL between January 2015 and January 2017 at eight units (Phase 1), with two centers continuing recruitment between January 2017 and March 2021 (Phase 2). Serum samples were collected routinely and sent for β-human chorionic gonadotropin (β-hCG) and progesterone measurement. Clinical factors recorded were maternal age, pain score, bleeding score and history of EP. Based on transvaginal ultrasonography and/or biochemical confirmation during follow-up, PUL were classified subsequently as failed PUL (FPUL), intrauterine pregnancy (IUP) or EP (including persistent PUL (PPUL)). The M6 models with (M6P ) and without (M6NP ) progesterone were refitted and extended with clinical factors. Model validation was performed using internal-external cross-validation (IECV) (Phase 1) and temporal external validation (EV) (Phase 2). Missing values were handled using multiple imputation.

RESULTS

Overall, 5473 PUL were recruited over both phases. A total of 709 PUL were excluded because maternal age was < 16 years or initial β-hCG was ≤ 25 IU/L, leaving 4764 (87%) PUL for analysis (2894 in Phase 1 and 1870 in Phase 2). For the refitted M6P model, the area under the receiver-operating-characteristics curve (AUC) for EP/PPUL vs IUP/FPUL was 0.89 for IECV and 0.84-0.88 for EV, with respective sensitivities of 94% and 92-93%. For the refitted M6NP model, the AUCs were 0.85 for IECV and 0.82-0.86 for EV, with respective sensitivities of 92% and 93-94%. Calibration performance was good overall, but with heterogeneity between centers. Net Benefit confirmed clinical utility. The change in AUC when M6P was extended to include maternal age, bleeding score and history of EP was between -0.02 and 0.01, depending on center and phase. The corresponding change in AUC when M6NP was extended was between -0.01 and 0.03. At the 5% threshold to define high risk of EP/PPUL, extending M6P altered sensitivity by -0.02 to -0.01, specificity by 0.03 to 0.04 and Net Benefit by -0.005 to 0.006. Extending M6NP altered sensitivity by -0.03 to -0.01, specificity by 0.05 to 0.07 and Net Benefit by -0.005 to 0.006.

CONCLUSIONS

The updated M6 model offers accurate diagnostic performance, with excellent sensitivity for EP. Adding clinical factors to the model improved performance in some centers, especially when progesterone levels were not suitable or unavailable. © 2023 The Authors. Ultrasound in Obstetrics & Gynecology published by John Wiley & Sons Ltd on behalf of International Society of Ultrasound in Obstetrics and Gynecology.

Development of a single-visit protocol for the management of pregnancy of unknown location following in vitro fertilization: a retrospective study

STUDY QUESTION

Can women with pregnancy of unknown location (PUL) following in vitro fertilization (IVF) be risk-stratified regarding the subsequent need for medical intervention, based on their demographic characteristics and the results of serum biochemistry at the initial visit?

SUMMARY ANSWER

The ratio of serum hCG to number of days from conception (hCG/C) or the initial serum hCG level at ≥5 weeks' gestation could be used to estimate the risk of women presenting with PUL following IVF and needing medical intervention during their follow-up.

WHAT IS KNOWN ALREADY

In women with uncertain conception dates presenting with PUL, a single serum hCG measurement cannot be used to predict the final pregnancy outcomes, thus, serial levels are mandatory to establish a correct diagnosis. Serum progesterone levels can help to risk-stratify women at their initial visit but are not accurate in those taking progesterone supplementation, such as women pregnant following IVF.

STUDY DESIGN, SIZE, DURATION

This was a retrospective study carried out at two specialist early pregnancy assessment units between May 2008 and January 2021. A total of 224 women met the criteria for inclusion, but 14 women did not complete the follow-up and were excluded from the study.

PARTICIPANTS/MATERIALS, SETTING, METHODS

We selected women who had an IVF pregnancy and presented with PUL at ≥5 weeks' gestation.

MAIN RESULTS AND THE ROLE OF CHANCE

A total of 30/210 (14.0%, 95% CI 9.9-19.8) women initially diagnosed with PUL required surgical intervention. The hCG/C was significantly higher in the group of women requiring an intervention compared to those who did not (P = 0.003), with an odds ratio of 3.65 (95% CI 1.49-8.89, P = 0.004). A hCG/C <4.0 was associated with a 1.9% risk of intervention, which accounted for 25.7% of the study population. A similar result was obtained by substituting hCG/C <4.0 with an initial hCG level <100 IU/l, which was associated with 2.0% risk of intervention, and accounted for 23.8% of the study population (P > 0.05).

LIMITATIONS, REASONS FOR CAUTION

A limitation of our study is that it is retrospective in nature, and as such, we were reliant on existing data.

WIDER IMPLICATIONS OF THE FINDINGS

A previous study in women with PUL after spontaneous conception found that a 2% intervention rate was considered low enough to eliminate the need for close follow-up and serial blood tests. Using the same 2% cut-off, a quarter of women with PUL after IVF could also avoid attending for further visits and investigations.

STUDY FUNDING/COMPETING INTEREST(S)

No external funding was required for this study. No conflicts of interest are required to be declared.

TRIAL REGISTRATION NUMBER

N/A.

Beta-human chorionic gonadotrophin point of care testing for the management of pregnancy of unknown location

RESEARCH QUESTION

Does a commercially available quantitative beta-human chorionic gonadotrophin (BHCG) point of care testing (POCT) device improve workflow management in early pregnancy by performing comparably to gold standard laboratory methods, and is the performance of a validated pregnancy of unknown location (PUL) triage strategy maintained using POCT BHCG results?

DESIGN

Women classified with a PUL between 2018 and 2021 at three early pregnancy units were included. The linear relationship of untreated whole-blood POCT and serum laboratory BHCG values was defined using coefficients and regression. Paired serial BHCG values were then incorporated into the validated M6 multinomial logistic regression model to stratify the PUL as at high risk or at low risk of clinical complications. The sensitivity and negative predictive value were assessed. The timings required for equivocal POCT and laboratory care pathways were compared.

RESULTS

A total of 462 PUL were included. The discrepancy between 571 laboratory and POCT BHCG values was -5.2% (-6.2 IU/l), with a correlation coefficient of 0.96. The 133 PUL with paired 0 and 48 h BHCG values were compared using the M6 model. The sensitivity for high-risk outcomes (96.2%) and negative predictive values (98.5%) was excellent for both. Sample receipt and laboratory processing took 135 min (421 timings), compared with 12 min (91 timings) when using POCT (P < 0.0001).

CONCLUSIONS

POCT BHCG values correlated well with laboratory testing measurements. The M6 model retained its performance when using POCT BHCG values. Using the model with POCT may improve workflow and patient care without compromising on effective PUL triage.

Follow-up and outcomes of patients with a pregnancy of unknown location: A comparison of two prediction models

BACKGROUND

The time period while delineating the final diagnosis following presentation with a pregnancy of unknown location (PUL) can be an anxious time, as well as being time and resource intensive. Prediction models have been utilised in order to tailor counselling, frame expectations and plan care.

OBJECTIVES

We aimed to review diagnoses of PUL in our population and assess the value of two prediction models.

STUDY DESIGN

We reviewed all 394 PUL diagnoses over a three year period in a tertiary level maternity hospital. We then retrospectively applied the M1 and M6NP models to assess their accuracy when compared to the final diagnosis.

RESULTS

PUL comprises of 2.9% (394/13401) of attendances in our unit, requiring 752 scans and 1613 separate blood tests. Just under one in ten women (9.9%, n = 39) presenting with a PUL had a viable pregnancy at discharge, however of the remainder, only 18.0% (n = 83) required medical or surgical treatment for a PUL. The M1 model was more successful at predicting an ectopic pregnancy than the M6NP, with the latter over-predicting viable pregnancies (33.4%, n = 77).

CONCLUSIONS

We demonstrate that the management of women with a PUL could be stratified through the application of outcome prediction models, having positive results for framing expectations and potentially reducing this resource-intensive diagnosis.

Pregnancy of unknown location: external validation of the hCG-based M6NP and M4 prediction models in an emergency gynaecology unit

OBJECTIVE

To investigate if M6NP predicting ectopic pregnancy (EP) among women with pregnancy of unknown location (PUL) is valid in an emergency gynaecology setting and comparing it with its predecessor M4.

DESIGN

Retrospective cohort study.

SETTING

University Hospital.

PARTICIPANTS

Women with PUL.

METHODS

All consecutive women with a PUL during a study period of 3 years were screened for inclusion. Risk prediction of an EP was based on two serum human chorionic gonadotropin (hCG) levels taken at least 24 hours and no longer than 72 hours apart.

MAIN OUTCOME MEASURES

The area under the ROC curve (AUC) expressed the ability of a model to distinguish an EP from a non-EP (discrimination). Calibration assessed the agreement between the predicted risk of an EP and the true risk (proportion) of EP. The proportion of EPs and non-EPs classified as high risk assessed the model's sensitivity and false positive rate (FPR). The proportion of non-EPs among women classified as low risk was the model's negative predictive value (NPV). The clinical utility of a model was evaluated with decision curve analysis.

RESULTS

1061 women were included in the study, of which 238 (22%) had a final diagnosis of EP. The AUC for EP was 0.85 for M6NP and 0.81 for M4. M6NP made accurate risk predictions of EP up to predictions of 20% but thereafter risks were underestimated. M4 was poorly calibrated up to risk predictions of 40%. With a 5% threshold for high risk classification the sensitivity for EP was 95% for M6NP, the FPR 50% and NPV 97%. M6NP had higher sensitivity and NPV than M4 but also a higher FPR. M6NP had utility at all thresholds as opposed to M4 that had no utility at thresholds≤5%.

CONCLUSIONS

M6NP had better predictive performance than M4 and is valid in women with PUL attending an emergency gynaecology unit. Our results can encourage implementation of M6NP in related yet untested clinical settings to effectively support clinical decision-making.

[Retrospective validation of a model to predict the outcome of pregnancies of unknown location]

OBJECTIVE

The prediction model M6 classifies pregnancy of unknown location (PUL) into a low-risk or a high-risk group in developing ectopic pregnancy (EP). The aim of this study was to validate the two-step M6 model's ability to classify PUL in French women.

MATERIAL AND METHODS

All women with a diagnosis of PUL over a year were included in this single center retrospective study. Patients with a diagnosis of EP at the first consultation of with incomplete data were excluded. For each patient, the M6 model calculator was used to classified them into "high risk of EP" and "low risk of EP" group. The reference standard was the final diagnostic: failed PUL (FPUL), intrauterine pregnancy (IUP) of EP. The statistical measures of the test's performance were calculated.

RESULTS

Over the period, 255 women's consulted for a PUL, 197 has been included in the study. Final diagnosis were: 94 FPUL (94/197; 47.7%), 74 IUP (74/197; 37.6%) et 29 EP (29/197; 14.7%). The first step of the M6 model classified 16 women in the FPUL group of which 15 (15/16; 93.7%) correctly. The second step of the M6 model classified 181 women: 90 (90/181; 49.7%) in the "high risk of EP" group of which 63 (63/90; 70%) were FPUL/IUP and 27 (27/90; 30%) were EP. 91 (91/181; 50.3%) was classified in the "low risk of EP" group of which 90 (90/91; 98.9%) were FPUL/IUP and 1 (1/91; 1.1%) were EP. EP were correctly classified with sensitivity of 96.4%, negative predictive value of 98.9%, specificity of 58.8% and positive predictive value of 30.0%.

CONCLUSIONS

The prediction model of PUL M6 classified EP in "high risk of EP group" with a sensitivity of 96.4%. It classified 50.3% of PUL in a "low risk of EP" group with a negative predictive value of 98.9%.

Ultrasound characteristics, serum biochemistry and outcome of ectopic pregnancies presenting during COVID-19 pandemic

OBJECTIVE

To describe and compare the characteristics of ectopic pregnancies (EPs) in the year prior to vs during the coronavirus disease 2019 (COVID-19) pandemic.

METHODS

This was a retrospective analysis of women diagnosed with an EP on transvaginal sonography conducted at a center in London, UK, providing early-pregnancy assessment, between 1 January 2019 and 31 December 2020. Women were identified via the Astraia ultrasound reporting system using coded and non-coded outcomes of EP or pregnancy outside the uterine cavity. Data related to predefined outcomes were collected using Astraia and Cerner electronic reporting systems. Main outcome measures included clinical, ultrasound and biochemical features of EP, in addition to reported complications and management.

RESULTS

There were 22 683 consultations over the 2-year period. Following consultation, a similar number and proportion of EPs were diagnosed in 2019 (141/12 657 (1%)) and 2020 (134/10 026 (1%)). Both cohorts were comparable in age, ethnicity, weight and method of conception. Gestational age at the first transvaginal sonography scan and at diagnosis were similar, and no difference in location, size or morphology of EP was found between the two cohorts. Serum human chorionic gonadotropin (hCG) levels at the time of EP diagnosis were higher in 2020 than in 2019 (1005 IU/L vs 665 IU/L; P = 0.03). The proportions of women according to type of final EP management were similar, but the rate of failed first-line management was higher during vs before the pandemic (16% vs 6%; P = 0.01). The rates of blood detected in the pelvis (hemoperitoneum) on ultrasound (23% vs 26%; P = 0.58) and of ruptured EP confirmed surgically (9% vs 3%; P = 0.07) were similar in 2019 vs 2020.

CONCLUSIONS

No difference was observed in the location, size, morphology or gestational age at the first ultrasound examination or at diagnosis of EP between women diagnosed before vs during the COVID-19 pandemic. Complication rates and final management strategy were also unchanged. However, hCG levels and the failure rate of first-line conservative management measures were higher during the pandemic. Our findings suggest that women continued to access appropriate care for EP during the COVID-19 pandemic, with no evidence of diagnostic delay or an increase in adverse outcome in our population. © 2021 International Society of Ultrasound in Obstetrics and Gynecology.

Evaluating cut-off levels for progesterone, β human chorionic gonadotropin and β human chorionic gonadotropin ratio to exclude pregnancy viability in women with a pregnancy of unknown location: A prospective multicenter cohort study

INTRODUCTION

There is no global agreement on how to best determine pregnancy of unknown location viability and location using biomarkers. Measurements of progesterone and β human chorionic gonadotropin (βhCG) are still used in clinical practice to exclude the possibility of a viable intrauterine pregnancy (VIUP). We evaluate the predictive value of progesterone, βhCG, and βhCG ratio cut-off levels to exclude a VIUP in women with a pregnancy of unknown location.

MATERIAL AND METHODS

This was a secondary analysis of prospective multicenter study data of consecutive women with a pregnancy of unknown location between January 2015 and 2017 collected from dedicated early pregnancy assessment units of eight hospitals. Single progesterone and serial βhCG measurements were taken. Women were followed up until final pregnancy outcome between 11 and 14 weeks of gestation was confirmed using transvaginal ultrasonography: (1) VIUP, (2) non-viable intrauterine pregnancy or failed pregnancy of unknown location, and (3) ectopic pregnancy or persisting pregnancy of unknown location. The predictive value of cut-off levels for ruling out VIUP were evaluated across a range of values likely to be encountered clinically for progesterone, βhCG, and βhCG ratio.

RESULTS

Data from 2507 of 3272 (76.6%) women were suitable for analysis. All had data for βhCG levels, 2248 (89.7%) had progesterone levels, and 1809 (72.2%) had βhCG ratio. The likelihood of viability falls with the progesterone level. Although the median progesterone level associated with viability was 59 nmol/L, VIUP were identified with levels as low as 5 nmol/L. No single βhCG cut-off reliably ruled out the presence of viability with certainty, even when the level was more than 3000 IU/L, there were 39/358 (11%) women who had a VIUP. The probability of viability decreases with the βhCG ratio. Although the median βhCG ratio associated with viability was 2.26, VIUP were identified with ratios as low as 1.02. A progesterone level below 2 nmol/L and βhCG ratio below 0.87 were unlikely to be associated with viability but were not definitive when considering multiple imputation.

CONCLUSIONS

Cut-off levels for βhCG, βhCG ratio, and progesterone are not safe to be used clinically to exclude viability in early pregnancy. Although βhCG ratio and progesterone have slightly better performance in comparison, single βhCG used in this manner is highly unreliable.

External validation of models to predict the outcome of pregnancies of unknown location: a multicentre cohort study

Objective

To validate externally five approaches to predict ectopic pregnancy (EP) in pregnancies of unknown location (PUL): the M6P and M6NP risk models, the two‐step triage strategy (2ST, which incorporates M6P), the M4 risk model, and beta human chorionic gonadotropin ratio cut‐offs (BhCG‐RC).

Design

Secondary analysis of a prospective cohort study.

Setting

Eight UK early pregnancy assessment units.

Population

Women presenting with a PUL and BhCG >25 IU/l.

Methods

Women were managed using the 2ST protocol: PUL were classified as low risk of EP if presenting progesterone ≤2 nmol/l; the remaining cases returned 2 days later for triage based on M6P. EP risk ≥5% was used to classify PUL as high risk. Missing values were imputed, and predictions for the five approaches were calculated post hoc. We meta‐analysed centre‐specific results.

Main outcome measures

Discrimination, calibration and clinical utility (decision curve analysis) for predicting EP.

Results

Of 2899 eligible women, the primary analysis excluded 297 (10%) women who were lost to follow up. The area under the ROC curve for EP was 0.89 (95% CI 0.86–0.91) for M6P, 0.88 (0.86–0.90) for 2ST, 0.86 (0.83–0.88) for M6NP and 0.82 (0.78–0.85) for M4. Sensitivities for EP were 96% (M6P), 94% (2ST), 92% (N6NP), 80% (M4) and 58% (BhCG‐RC); false‐positive rates were 35%, 33%, 39%, 24% and 13%. M6P and 2ST had the best clinical utility and good overall calibration, with modest variability between centres.

Conclusions

2ST and M6P performed best for prediction and triage in PUL.

Tweetable abstract

The M6 model, as part of a two‐step triage strategy, is the best approach to characterise and triage PULs.

The M6 model, as part of a two‐step triage strategy, is the best approach to characterise and triage PULs.