Prediction of obstetrical and fetal complications using automated electronic health record data

Prediction of neonatal outcomes using gestational age vs ACOG definitions of maternal disease severity in hypertensive disorders of pregnancy

PURPOSE

Hypertensive disorders of pregnancy cause significant neonatal complications. Disease severity is often used to predict neonatal outcomes, however gestational age (GA) at delivery may be a better predictor. We aimed to assess whether disease severity or GA was more predictive of adverse neonatal outcomes.

METHODS

We included 165 participants with confirmed HELLP syndrome or severe preeclampsia (sPE). Two predictive models were constructed to assess the ability of disease severity compared to GA to predict a composite adverse neonatal outcome. The composite outcome included low birth weight, SGA, IUGR, Apgar score, and neonatal death.

RESULTS

Using severity as a predictor of binary neonatal outcome had an AUC of 0.73 (0.65-0.81), with a sensitivity (SE) of 70.3% and a specificity (SP) of 64.4%. For GA, we observed an AUC of 0.82 (0.75-0.89), with a SE of 75.7% and a SP of 76.7%.

CONCLUSION

For the composite neonatal outcome, GA was a better predictor than ACOG diagnosis (severity). This observation underscores the need for further research to validate these findings in larger cohorts and to determine their applicability to maternal outcomes.

Development and Validation of an Automated, Real-Time Predictive Model for Postpartum Hemorrhage

OBJECTIVE

To develop and validate a predictive model for postpartum hemorrhage that can be deployed in clinical care using automated, real-time electronic health record (EHR) data and to compare performance of the model with a nationally published risk prediction tool.

METHODS

A multivariable logistic regression model was developed from retrospective EHR data from 21,108 patients delivering at a quaternary medical center between January 1, 2018, and April 30, 2022. Deliveries were divided into derivation and validation sets based on an 80/20 split by date of delivery. Postpartum hemorrhage was defined as blood loss of 1,000 mL or more in addition to postpartum transfusion of 1 or more units of packed red blood cells. Model performance was evaluated by the area under the receiver operating characteristic curve (AUC) and was compared with a postpartum hemorrhage risk assessment tool published by the CMQCC (California Maternal Quality Care Collaborative). The model was then programmed into the EHR and again validated with prospectively collected data from 928 patients between November 7, 2023, and January 31, 2024.

RESULTS

Postpartum hemorrhage occurred in 235 of 16,862 patients (1.4%) in the derivation cohort. The predictive model included 21 risk factors and demonstrated an AUC of 0.81 (95% CI, 0.79-0.84) and calibration slope of 1.0 (Brier score 0.013). During external temporal validation, the model maintained discrimination (AUC 0.80, 95% CI, 0.72-0.84) and calibration (calibration slope 0.95, Brier score 0.014). This was superior to the CMQCC tool (AUC 0.69 [95% CI, 0.67-0.70], P <.001). The model maintained performance in prospective, automated data collected with the predictive model in real time (AUC 0.82 [95% CI, 0.73-0.91]).

CONCLUSION

We created and temporally validated a postpartum hemorrhage prediction model, demonstrated its superior performance over a commonly used risk prediction tool, successfully coded the model into the EHR, and prospectively validated the model using risk factor data collected in real time. Future work should evaluate external generalizability and effects on patient outcomes; to facilitate this work, we have included the model coefficients and examples of EHR integration in the article.

An all-inclusive model for predicting invasive bacterial infection in febrile infants age 7-60 days

BACKGROUND

Invasive bacterial infections (IBIs) in febrile infants are rare but potentially devastating. We aimed to derive and validate a predictive model for IBI among febrile infants age 7-60 days.

METHODS

Data were abstracted retrospectively from electronic records of 37 emergency departments (EDs) for infants with a measured temperature >=100.4 F who underwent an ED evaluation with blood and urine cultures. Models to predict IBI were developed and validated respectively using a random 80/20 dataset split, including 10-fold cross-validation. We used precision recall curves as the classification metric.

RESULTS

Of 4411 eligible infants with a mean age of 37 days, 29% had characteristics that would likely have excluded them from existing risk stratification protocols. There were 196 patients with IBI (4.4%), including 43 (1.0%) with bacterial meningitis. Analytic approaches varied in performance characteristics (precision recall range 0.04-0.29, area under the curve range 0.5-0.84), with the XGBoost model demonstrating the best performance (0.29, 0.84). The five most important variables were serum white blood count, maximum temperature, absolute neutrophil count, absolute band count, and age in days.

CONCLUSION

A machine learning model (XGBoost) demonstrated the best performance in predicting a rare outcome among febrile infants, including those excluded from existing algorithms.

IMPACT

Several models for the risk stratification of febrile infants have been developed. There is a need for a preferred comprehensive model free from limitations and algorithm exclusions that accurately predicts IBIs. This is the first study to derive an all-inclusive predictive model for febrile infants aged 7-60 days in a community ED sample with IBI as a primary outcome. This machine learning model demonstrates potential for clinical utility in predicting IBI.

External Validation of Postpartum Hemorrhage Prediction Models Using Electronic Health Record Data

OBJECTIVE

A recent study leveraging machine learning methods found that postpartum hemorrhage (PPH) can be predicted accurately at the time of labor admission in the U.S. Consortium for Safe Labor (CSL) dataset, with a C-statistic as high as 0.93. These CSL models were developed in older data (2002-2008) and used an estimated blood loss (EBL) of ≥1,000 mL to define PPH. We sought to externally validate these models using a more recent cohort of births where blood loss was measured using quantitative blood loss (QBL) methods.

STUDY DESIGN

Using data from 5,261 deliveries between February 1, 2019 and May 11, 2020 at a single tertiary hospital, we mapped our electronic health record (EHR) data to the 55 predictors described in previously published CSL models. PPH was defined as QBL ≥1,000 mL within 24 hours after delivery. Model discrimination and calibration of the four CSL models were measured using our cohort. In a secondary analysis, we fit new models in our study cohort using the same predictors and algorithms as the original CSL models.

RESULTS

The original study cohort had a substantially lower rate of PPH, 4.8% (7,279/228,438) versus 25% (1,321/5,261), possibly due to differences in measurement. The CSL models had lower discrimination in our study cohort, with a C-statistic as high as 0.57 (logistic regression). Models refit in our study cohort achieved better discrimination, with a C-statistic as high as 0.64 (random forest). Calibration improved in the refit models as compared with the original models.

CONCLUSION

The CSL models' accuracy was lower in a contemporary EHR where PPH is assessed using QBL. As institutions continue to adopt QBL methods, further data are needed to understand the differences between EBL and QBL to enable accurate prediction of PPH.

KEY POINTS

· Machine learning methods may help predict PPH.. · EBL models do not generalize when QBL is used.. · Blood loss estimation alters model accuracy..

Artificial intelligence as a new answer to old challenges in maternal-fetal medicine and obstetrics

BACKGROUND

Following the latest trends in the development of artificial intelligence (AI), the possibility of processing an immense amount of data has created a breakthrough in the medical field. Practitioners can now utilize AI tools to advance diagnostic protocols and improve patient care.

OBJECTIVE

The aim of this article is to present the importance and modalities of AI in maternal-fetal medicine and obstetrics and its usefulness in daily clinical work and decision-making process.

METHODS

A comprehensive literature review was performed by searching PubMed for articles published from inception up until August 2023, including the search terms "artificial intelligence in obstetrics", "maternal-fetal medicine", and "machine learning" combined through Boolean operators. In addition, references lists of identified articles were further reviewed for inclusion.

RESULTS

According to recent research, AI has demonstrated remarkable potential in improving the accuracy and timeliness of diagnoses in maternal-fetal medicine and obstetrics, e.g., advancing perinatal ultrasound technique, monitoring fetal heart rate during labor, or predicting mode of delivery. The combination of AI and obstetric ultrasound can help optimize fetal ultrasound assessment by reducing examination time and improving diagnostic accuracy while reducing physician workload.

CONCLUSION

The integration of AI in maternal-fetal medicine and obstetrics has the potential to significantly improve patient outcomes, enhance healthcare efficiency, and individualized care plans. As technology evolves, AI algorithms are likely to become even more sophisticated. However, the successful implementation of AI in maternal-fetal medicine and obstetrics needs to address challenges related to interpretability and reliability.

Postpartum readmission for hypertension and pre-eclampsia: development and validation of a predictive model

OBJECTIVE

To develop a model for predicting postpartum readmission for hypertension and pre-eclampsia at delivery discharge and assess external validation or model transportability across clinical sites.

DESIGN

Prediction model using data available in the electronic health record from two clinical sites.

SETTING

Two tertiary care health systems from the Southern (2014-2015) and Northeastern USA (2017-2019).

POPULATION

A total of 28 201 postpartum individuals: 10 100 in the South and 18 101 in the Northeast.

METHODS

An internal-external cross validation (IECV) approach was used to assess external validation or model transportability across the two sites. In IECV, data from each health system were first used to develop and internally validate a prediction model; each model was then externally validated using the other health system. Models were fit using penalised logistic regression, and accuracy was estimated using discrimination (concordance index), calibration curves and decision curves. Internal validation was performed using bootstrapping with bias-corrected performance measures. Decision curve analysis was used to display potential cut points where the model provided net benefit for clinical decision-making.

MAIN OUTCOME MEASURES

The outcome was postpartum readmission for either hypertension or pre-eclampsia <6 weeks after delivery.

RESULTS

The postpartum readmission rate for hypertension and pre-eclampsia overall was 0.9% (0.3% and 1.2% by site, respectively). The final model included six variables: age, parity, maximum postpartum diastolic blood pressure, birthweight, pre-eclampsia before discharge and delivery mode (and interaction between pre-eclampsia × delivery mode). Discrimination was adequate at both health systems on internal validation (c-statistic South: 0.88; 95% confidence interval [CI] 0.87-0.89; Northeast: 0.74; 95% CI 0.74-0.74). In IECV, discrimination was inconsistent across sites, with improved discrimination for the Northeastern model on the Southern cohort (c-statistic 0.61 and 0.86, respectively), but calibration was not adequate. Next, model updating was performed using the combined dataset to develop a new model. This final model had adequate discrimination (c-statistic: 0.80, 95% CI 0.80-0.80), moderate calibration (intercept -0.153, slope 0.960, Emax 0.042) and provided superior net benefit at clinical decision-making thresholds between 1% and 7% for interventions preventing readmission. An online calculator is provided here.

CONCLUSIONS

Postpartum readmission for hypertension and pre-eclampsia may be accurately predicted but further model validation is needed. Model updating using data from multiple sites will be needed before use across clinical settings.

Anger: an underappreciated destructive force in healthcare

Anger is an emotional state that occurs when unexpected things happen to or around oneself and is "an emotional state that varies in intensity from mild irritation to intense fury and rage." It is defined as "a strong feeling of displeasure and usually of antagonism," an emotion characterized by tension and hostility arising from frustration, real or imagined injury by another, or perceived injustice. It can manifest itself in behaviors designed to remove the object of the anger (e.g., determined action) or behaviors designed merely to express the emotion. For the Roman philosopher Seneca anger is not an uncontrollable, impulsive, or instinctive reaction. It is, rather, the cognitive assent that such initial reactions to the offending action or words are in fact unjustified. It is, rather, the cognitive assent that such initial reactions to the offending action or words are in fact unjustified. It seems that the year 2022 was a year when many Americans were plainly angry. "Why is everyone so angry?" the New York Times asked in the article "The Year We Lost It." We believe that Seneca is correct in that anger is unacceptable. Anger is a negative emotion that must be controlled, and Seneca provides us with the tools to avoid and destroy anger. Health care professionals will be more effective, content, and happier if they learn more about Seneca's writings about anger and implement his wisdom on anger from over 2000 years ago.

Multi-omics and machine learning for the prevention and management of female reproductive health

Females typically carry most of the burden of reproduction in mammals. In humans, this burden is exacerbated further, as the evolutionary advantage of a large and complex human brain came at a great cost of women's reproductive health. Pregnancy thus became a highly demanding phase in a woman's life cycle both physically and emotionally and therefore needs monitoring to assure an optimal outcome. Moreover, an increasing societal trend towards reproductive complications partly due to the increasing maternal age and global obesity pandemic demands closer monitoring of female reproductive health. This review first provides an overview of female reproductive biology and further explores utilization of large-scale data analysis and -omics techniques (genomics, transcriptomics, proteomics, and metabolomics) towards diagnosis, prognosis, and management of female reproductive disorders. In addition, we explore machine learning approaches for predictive models towards prevention and management. Furthermore, mobile apps and wearable devices provide a promise of continuous monitoring of health. These complementary technologies can be combined towards monitoring female (fertility-related) health and detection of any early complications to provide intervention solutions. In summary, technological advances (e.g., omics and wearables) have shown a promise towards diagnosis, prognosis, and management of female reproductive disorders. Systematic integration of these technologies is needed urgently in female reproductive healthcare to be further implemented in the national healthcare systems for societal benefit.

A Scoping Review of the Use of Machine Learning in Health Economics and Outcomes Research: Part 2-Data From Nonwearables

OBJECTIVES

Despite the increasing interest in applying machine learning (ML) methods in health economics and outcomes research (HEOR), stakeholders face uncertainties in when and how ML can be used. We reviewed the recent applications of ML in HEOR.

METHODS

We searched PubMed for studies published between January 2020 and March 2021 and randomly chose 20% of the identified studies for the sake of manageability. Studies that were in HEOR and applied an ML technique were included. Studies related to wearable devices were excluded. We abstracted information on the ML applications, data types, and ML methods and analyzed it using descriptive statistics.

RESULTS

We retrieved 805 articles, of which 161 (20%) were randomly chosen. Ninety-two of the random sample met the eligibility criteria. We found that ML was primarily used for predicting future events (86%) rather than current events (14%). The most common response variables were clinical events or disease incidence (42%) and treatment outcomes (22%). ML was less used to predict economic outcomes such as health resource utilization (16%) or costs (3%). Although electronic medical records (35%) were frequently used for model development, claims data were used less frequently (9%). Tree-based methods (eg, random forests and boosting) were the most commonly used ML methods (31%).

CONCLUSIONS

The use of ML techniques in HEOR is growing rapidly, but there remain opportunities to apply them to predict economic outcomes, especially using claims databases, which could inform the development of cost-effectiveness models.

Comparison of machine learning and logistic regression as predictive models for adverse maternal and neonatal outcomes of preeclampsia: A retrospective study

Introduction

Preeclampsia, one of the leading causes of maternal and fetal morbidity and mortality, demands accurate predictive models for the lack of effective treatment. Predictive models based on machine learning algorithms demonstrate promising potential, while there is a controversial discussion about whether machine learning methods should be recommended preferably, compared to traditional statistical models.

Methods

We employed both logistic regression and six machine learning methods as binary predictive models for a dataset containing 733 women diagnosed with preeclampsia. Participants were grouped by four different pregnancy outcomes. After the imputation of missing values, statistical description and comparison were conducted preliminarily to explore the characteristics of documented 73 variables. Sequentially, correlation analysis and feature selection were performed as preprocessing steps to filter contributing variables for developing models. The models were evaluated by multiple criteria.

Results

We first figured out that the influential variables screened by preprocessing steps did not overlap with those determined by statistical differences. Secondly, the most accurate imputation method is K-Nearest Neighbor, and the imputation process did not affect the performance of the developed models much. Finally, the performance of models was investigated. The random forest classifier, multi-layer perceptron, and support vector machine demonstrated better discriminative power for prediction evaluated by the area under the receiver operating characteristic curve, while the decision tree classifier, random forest, and logistic regression yielded better calibration ability verified, as by the calibration curve.

Conclusion

Machine learning algorithms can accomplish prediction modeling and demonstrate superior discrimination, while Logistic Regression can be calibrated well. Statistical analysis and machine learning are two scientific domains sharing similar themes. The predictive abilities of such developed models vary according to the characteristics of datasets, which still need larger sample sizes and more influential predictors to accumulate evidence.

Comparison of Natural Language Processing of Clinical Notes With a Validated Risk-Stratification Tool to Predict Severe Maternal Morbidity

IMPORTANCE

Risk-stratification tools are routinely used in obstetrics to assist care teams in assessing and communicating risk associated with delivery. Electronic health record data and machine learning methods may offer a novel opportunity to improve and automate risk assessment.

OBJECTIVE

To compare the predictive performance of natural language processing (NLP) of clinician documentation with that of a previously validated tool to identify individuals at high risk for maternal morbidity.

DESIGN, SETTING, AND PARTICIPANTS

This retrospective diagnostic study was conducted at Brigham and Women's Hospital and Massachusetts General Hospital, Boston, Massachusetts, and included individuals admitted for delivery at the former institution from July 1, 2016, to February 29, 2020. A subset of these encounters (admissions from February to December 2018) was part of a previous prospective validation study of the Obstetric Comorbidity Index (OB-CMI), a comorbidity-weighted score to stratify risk of severe maternal morbidity (SMM).

EXPOSURES

Natural language processing of clinician documentation and OB-CMI scores.

MAIN OUTCOMES AND MEASURES

Natural language processing of clinician-authored admission notes was used to predict SMM in individuals delivering at the same institution but not included in the prospective OB-CMI study. The NLP model was then compared with the OB-CMI in the subset with a known OB-CMI score. Model discrimination between the 2 approaches was compared using the DeLong test. Sensitivity and positive predictive value for the identification of individuals at highest risk were prioritized as the characteristics of interest.

RESULTS

This study included 19 794 individuals; 4034 (20.4%) were included in the original prospective validation study of the OB-CMI (testing set), and the remaining 15 760 (79.6%) composed the training set. Mean (SD) age was 32.3 (5.2) years in the testing cohort and 32.2 (5.2) years in the training cohort. A total of 115 individuals in the testing cohort (2.9%) and 468 in the training cohort (3.0%) experienced SMM. The NLP model was built from a pruned vocabulary of 2783 unique words that occurred within the 15 760 admission notes from individuals in the training set. The area under the receiver operating characteristic curve of the NLP-based model for the prediction of SMM was 0.76 (95% CI, 0.72-0.81) and was comparable with that of the OB-CMI model (0.74; 95% CI, 0.70-0.79) in the testing set (P = .53). Sensitivity (NLP, 28.7%; OB-CMI, 24.4%) and positive predictive value (NLP, 19.4%; OB-CMI, 17.6%) were comparable between the NLP and OB-CMI high-risk designations for the prediction of SMM.

CONCLUSIONS AND RELEVANCE

In this study, the NLP method and a validated risk-stratification tool had a similar ability to identify patients at high risk of SMM. Future prospective research is needed to validate the NLP approach in clinical practice and determine whether it could augment or replace tools requiring manual user input.

Development and evaluation of a nomogram for adverse outcomes of preeclampsia in Chinese pregnant women

Objective

Preeclampsia, the main cause of maternal and perinatal deaths, is associated with several maternal complications and adverse perinatal outcomes. Some prediction models are uesd to evaluate adverse pregnancy outcomes. However, some of the current prediction models are mainly carried out in developed countries, and many problems are still exist. We, thus, developed and validated a nomogram to predict the risk of adverse pregnancy outcomes of preeclampsia in Chinese pregnant women.

Methods

The clinical data of 720 pregnant women with preeclampsia in seven medical institutions in Chongqing from January 1, 2010, to December 31, 2020, were analyzed retrospectively. The patients were divided into two groups: 180 cases (25%) with adverse outcomes and 540 cases (75%) without adverse outcomes. The indicators were identified via univariate analysis. Logistic regression analysis was used to establish the prediction model, which was displayed by a nomogram. The performance of the nomogram was evaluated in terms of the area under the receiver operating characteristic (ROC) curve, calibration, and clinical utility.

Results

Univariate analysis showed that 24 indicators were significantly different (P < 0.05). Logistic regression analysis showed that gestational age, 24 h urine protein qualitative, and TT were significantly different (P < 0.05). The area under the ROC curve was 0.781 (95% CI 0.737–0.825) in training set and 0.777 (95% CI 0.689–0.865) in test set. The calibration curve of the nomogram showed good agreement between prediction and observation. The analysis of the clinical decision curve showed that the nomogram is of practical significance.

Conclusion

Our study identified gestational age, 24 h urine protein qualitative, and TT as risk factors for adverse outcomes of preeclampsia in pregnant women, and constructed a nomogram that can easily predict and evaluate the risk of adverse pregnancy outcomes in women with preeclampsia.

Vision for Improving Pregnancy Health: Innovation and the Future of Pregnancy Research

Understanding, predicting, and preventing pregnancy disorders have been a major research target. Nonetheless, the lack of progress is illustrated by research results related to preeclampsia and other hypertensive pregnancy disorders. These remain a major cause of maternal and infant mortality worldwide. There is a general consensus that the rate of progress toward understanding pregnancy disorders lags behind progress in other aspects of human health. In this presentation, we advance an explanation for this failure and suggest solutions. We propose that progress has been impeded by narrowly focused research training and limited imagination and innovation, resulting in the failure to think beyond conventional research approaches and analytical strategies. Investigations have been largely limited to hypothesis-generating approaches constrained by attempts to force poorly defined complex disorders into a single "unifying" hypothesis. Future progress could be accelerated by rethinking this approach. We advise taking advantage of innovative approaches that will generate new research strategies for investigating pregnancy abnormalities. Studies should begin before conception, assessing pregnancy longitudinally, before, during, and after pregnancy. Pregnancy disorders should be defined by pathophysiology rather than phenotype, and state of the art agnostic assessment of data should be adopted to generate new ideas. Taking advantage of new approaches mandates emphasizing innovation, inclusion of large datasets, and use of state of the art experimental and analytical techniques. A revolution in understanding pregnancy-associated disorders will depend on networks of scientists who are driven by an intense biological curiosity, a team spirit, and the tools to make new discoveries.

Intelligent systems in obstetrics and midwifery: Applications of machine learning

INTRODUCTION

Machine learning is increasingly utilized over recent years in order to develop models that represent and solve problems in a variety of domains, including those of obstetrics and midwifery. The aim of this systematic review was to analyze research studies on machine learning and intelligent systems applications in midwifery and obstetrics.

METHODS

A thorough literature review was performed in four electronic databases (PubMed, APA PsycINFO, SCOPUS, ScienceDirect). Only articles that discussed machine learning and intelligent systems applications in midwifery and obstetrics, were considered in this review. Selected articles were critically evaluated as for their relevance and a contextual synthesis was conducted.

RESULTS

Thirty-two articles were included in this systematic review as they met the inclusion and methodological criteria specified in this study. The results suggest that machine learning and intelligent systems have produced successful models and systems in a broad list of midwifery and obstetrics topics, such as diagnosis, pregnancy risk assessment, fetal monitoring, bladder tumor, etc.

CONCLUSIONS

This systematic review suggests that machine learning represents a very promising area of artificial intelligence for the development of practical and highly effective applications that can support human experts, as well the investigation of a wide range of exciting opportunities for further research.

Development and Validation of an Obstetric Comorbidity Risk Score for Clinical Use

Background: A comorbidity summary score may support early and systematic identification of women at high risk for adverse obstetric outcomes. The objective of this study was to conduct the initial development and validation of an obstetrics comorbidity risk score for automated implementation in the electronic health record (EHR) for clinical use. Methods: The score was developed and validated using EHR data for a retrospective cohort of pregnancies with delivery between 2010 and 2018 at Kaiser Permanente Northern California, an integrated health care system. The outcome used for model development consisted of adverse obstetric events from delivery hospitalization (e.g., eclampsia, hemorrhage, death). Candidate predictors included maternal age, parity, multiple gestation, and any maternal diagnoses assigned in health care encounters in the 12 months before admission for delivery. We used penalized regression for variable selection, logistic regression to fit the model, and internal validation for model evaluation. We also evaluated prenatal model performance at 18 weeks of pregnancy. Results: The development cohort (n = 227,405 pregnancies) had an outcome rate of 3.8% and the validation cohort (n = 41,683) had an outcome rate of 2.9%. Of 276 candidate predictors, 37 were included in the final model. The final model had a validation c-statistic of 0.72 (95% confidence interval [CI] 0.70-0.73). When evaluated at 18 weeks of pregnancy, discrimination was modestly diminished (c-statistic 0.68 [95% CI 0.67-0.70]). Conclusions: The obstetric comorbidity score demonstrated good discrimination for adverse obstetric outcomes. After additional appropriate validation, the score can be automated in the EHR to support early identification of high-risk women and assist efforts to ensure risk-appropriate maternal care.

Heuristics in the delivery room

[Figure: see text].