1
|
Gil MM, Cuenca-Gómez D, Rolle V, Pertegal M, Díaz C, Revello R, Adiego B, Mendoza M, Molina FS, Santacruz B, Ansbacher-Feldman Z, Meiri H, Martin-Alonso R, Louzoun Y, De Paco Matallana C. Validation of machine-learning model for first-trimester prediction of pre-eclampsia using cohort from PREVAL study. Ultrasound Obstet Gynecol 2024; 63:68-74. [PMID: 37698356 DOI: 10.1002/uog.27478] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/08/2023] [Revised: 07/28/2023] [Accepted: 08/15/2023] [Indexed: 09/13/2023]
Abstract
OBJECTIVE Effective first-trimester screening for pre-eclampsia (PE) can be achieved using a competing-risks model that combines risk factors from the maternal history with multiples of the median (MoM) values of biomarkers. A new model using artificial intelligence through machine-learning methods has been shown to achieve similar screening performance without the need for conversion of raw data of biomarkers into MoM. This study aimed to investigate whether this model can be used across populations without specific adaptations. METHODS Previously, a machine-learning model derived with the use of a fully connected neural network for first-trimester prediction of early (< 34 weeks), preterm (< 37 weeks) and all PE was developed and tested in a cohort of pregnant women in the UK. The model was based on maternal risk factors and mean arterial blood pressure (MAP), uterine artery pulsatility index (UtA-PI), placental growth factor (PlGF) and pregnancy-associated plasma protein-A (PAPP-A). In this study, the model was applied to a dataset of 10 110 singleton pregnancies examined in Spain who participated in the first-trimester PE validation (PREVAL) study, in which first-trimester screening for PE was carried out using the Fetal Medicine Foundation (FMF) competing-risks model. The performance of screening was assessed by examining the area under the receiver-operating-characteristics curve (AUC) and detection rate (DR) at a 10% screen-positive rate (SPR). These indices were compared with those derived from the application of the FMF competing-risks model. The performance of screening was poor if no adjustment was made for the analyzer used to measure PlGF, which was different in the UK and Spain. Therefore, adjustment for the analyzer used was performed using simple linear regression. RESULTS The DRs at 10% SPR for early, preterm and all PE with the machine-learning model were 84.4% (95% CI, 67.2-94.7%), 77.8% (95% CI, 66.4-86.7%) and 55.7% (95% CI, 49.0-62.2%), respectively, with the corresponding AUCs of 0.920 (95% CI, 0.864-0.975), 0.913 (95% CI, 0.882-0.944) and 0.846 (95% CI, 0.820-0.872). This performance was achieved with the use of three of the biomarkers (MAP, UtA-PI and PlGF); inclusion of PAPP-A did not provide significant improvement in DR. The machine-learning model had similar performance to that achieved by the FMF competing-risks model (DR at 10% SPR, 82.7% (95% CI, 69.6-95.8%) for early PE, 72.7% (95% CI, 62.9-82.6%) for preterm PE and 55.1% (95% CI, 48.8-61.4%) for all PE) without requiring specific adaptations to the population. CONCLUSIONS A machine-learning model for first-trimester prediction of PE based on a neural network provides effective screening for PE that can be applied in different populations. However, before doing so, it is essential to make adjustments for the analyzer used for biochemical testing. © 2023 International Society of Ultrasound in Obstetrics and Gynecology.
Collapse
Affiliation(s)
- M M Gil
- Department of Obstetrics and Gynecology, Hospital Universitario de Torrejón, Torrejón de Ardoz, Madrid, Spain
- Faculty of Medicine, Universidad Francisco de Vitoria, Pozuelo de Alarcón, Madrid, Spain
| | - D Cuenca-Gómez
- Department of Obstetrics and Gynecology, Hospital Universitario de Torrejón, Torrejón de Ardoz, Madrid, Spain
- Faculty of Medicine, Universidad Francisco de Vitoria, Pozuelo de Alarcón, Madrid, Spain
| | - V Rolle
- Department of Obstetrics and Gynecology, Hospital Universitario de Torrejón, Torrejón de Ardoz, Madrid, Spain
- Clinical Research Unit, Hospital Universitario de Torrejón, Torrejón de Ardoz, Madrid, Spain
| | - M Pertegal
- Department of Obstetrics and Gynecology, Hospital Clínico Universitario 'Virgen de la Arrixaca', El Palmar, Murcia, Spain
- Institute for Biomedical Research of Murcia, IMIB-Arrixaca, El Palmar, Murcia, Spain
- Faculty of Medicine, Universidad de Murcia, Murcia, Spain
| | - C Díaz
- Department of Obstetrics and Gynecology, Complejo Hospitalario Universitario A Coruña, A Coruña, Galicia, Spain
| | - R Revello
- Department of Obstetrics and Gynecology, Hospital Universitario Quirón, Pozuelo de Alarcón, Madrid, Spain
| | - B Adiego
- Obstetrics and Gynecology Department, Hospital Universitario Fundación Alcorcón, Alcorcón, Madrid, Spain
| | - M Mendoza
- Department of Obstetrics and Gynecology, Hospital Universitari Vall d'Hebrón, Barcelona, Catalonia, Spain
| | - F S Molina
- Department of Obstetrics and Gynecology, Hospital Universitario San Cecilio, Granada, Spain
- Instituto de Investigación Biosanitaria (Ibs.GRANADA), Granada, Spain
| | - B Santacruz
- Department of Obstetrics and Gynecology, Hospital Universitario de Torrejón, Torrejón de Ardoz, Madrid, Spain
- Faculty of Medicine, Universidad Francisco de Vitoria, Pozuelo de Alarcón, Madrid, Spain
| | | | - H Meiri
- The ASPRE Consortium and TeleMarpe, Tel Aviv, Israel
| | - R Martin-Alonso
- Department of Obstetrics and Gynecology, Hospital Universitario de Torrejón, Torrejón de Ardoz, Madrid, Spain
- Faculty of Medicine, Universidad Francisco de Vitoria, Pozuelo de Alarcón, Madrid, Spain
| | - Y Louzoun
- Department of Mathematics, Bar Ilan University, Ramat Gan, Israel
| | - C De Paco Matallana
- Department of Obstetrics and Gynecology, Hospital Clínico Universitario 'Virgen de la Arrixaca', El Palmar, Murcia, Spain
- Institute for Biomedical Research of Murcia, IMIB-Arrixaca, El Palmar, Murcia, Spain
- Faculty of Medicine, Universidad de Murcia, Murcia, Spain
| |
Collapse
|
2
|
Ansbacher-Feldman Z, Syngelaki A, Meiri H, Cirkin R, Nicolaides KH, Louzoun Y. Machine-learning-based prediction of pre-eclampsia using first-trimester maternal characteristics and biomarkers. Ultrasound Obstet Gynecol 2022; 60:739-745. [PMID: 36454636 DOI: 10.1002/uog.26105] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/06/2022] [Revised: 08/01/2022] [Accepted: 08/03/2022] [Indexed: 06/17/2023]
Abstract
OBJECTIVE To evaluate the accuracy of predicting the risk of developing pre-eclampsia (PE) according to first-trimester maternal demographic characteristics, medical history and biomarkers using artificial-intelligence and machine-learning methods. METHODS The data were derived from prospective non-interventional screening for PE at 11-13 weeks' gestation at two maternity hospitals in the UK. The data were divided into three subsets. The first set, including 30 437 subjects, was used to develop the training process, the second set of 10 000 subjects was utilized to optimize the machine-learning hyperparameters and the third set of 20 352 subjects was coded and used for model validation. An artificial neural network was used to predict from the demographic characteristics and medical history the prior risk that was then combined with biomarker values to determine the risk of PE and preterm PE with delivery at < 37 weeks' gestation. An additional network was trained without including race as input. Biomarkers included uterine artery pulsatility index (UtA-PI), mean arterial blood pressure (MAP), placental growth factor (PlGF) and pregnancy-associated plasma protein-A. All markers were entered using raw values without conversion into standardized multiples of the median. The prediction accuracy was estimated using the area under the receiver-operating-characteristics curve (AUC). We further computed the detection rate at 10%, 20% and 40% false-positive rates (FPR). The impact of taking aspirin was also added. Shapley values were calculated to evaluate the contribution of each parameter to the prediction of risk. We used a non-parametric test to compare the expected AUC with the one obtained when we randomly scrambled the labels and kept the predictions. For the general prediction, we performed 10 000 permutations of the labels. When the AUC was higher than the one obtained in all 10 000 permutations, we reported a P-value of < 0.0001. For the race-specific analysis, we performed 1000 permutations. When the AUC was higher than the AUC in permutations, we reported a P-value of < 0.001. RESULTS The detection rate for preterm PE vs no PE, at a 10% FPR, was 53.3% when screening by maternal factors only, and the corresponding AUC was 0.816; these increased to 75.3% and 0.909, respectively, with the addition of biomarkers into the model. Information on race was important for the prediction accuracy; when race was not used to train the model, at a 10% FPR, the detection rate of preterm PE vs no PE decreased to 34.5-45.5% (for different races) when screening by maternal factors only and to 55.0-62.1% when biomarkers were added. The major predictors of PE were high MAP and UtA-PI, and low PlGF. The accuracy of prediction of all PE cases was lower than that for preterm PE. Aspirin use was recommended for cases who were at high risk of preterm PE. The AUC of all PE vs no PE was 0.770 when screening by maternal factors and 0.817 when the biomarkers were added; the respective detection rates, at a 10% FPR, were 41.3% and 52.9%. CONCLUSIONS Screening for PE using a non-linear machine-learning-based approach does not require a population-based normalization, and its performance is similar to that of logistic regression. Removing race information from the model reduces its prediction accuracy, especially for the non-white populations when only maternal factors are considered. © 2022 International Society of Ultrasound in Obstetrics and Gynecology.
Collapse
Affiliation(s)
| | - A Syngelaki
- Fetal Medicine Research Institute, King's College Hospital, London, UK
| | - H Meiri
- The ASPRE Consortium and TeleMarpe, Tel Aviv, Israel
| | - R Cirkin
- Department of Mathematics, Bar Ilan University, Ramat Gan, Israel
| | - K H Nicolaides
- Fetal Medicine Research Institute, King's College Hospital, London, UK
| | - Y Louzoun
- Department of Mathematics, Bar Ilan University, Ramat Gan, Israel
| |
Collapse
|