Use of artificial intelligence in obstetrics: not quite ready for prime time

Automatic Identification of Fetal Abdominal Planes from Ultrasound Images Based on Deep Learning

Fetal biometric assessments through ultrasound diagnostics are integral in obstetrics and gynecology, requiring considerable time investment. This study aimed to explore the potential of artificial intelligence (AI) architectures in automatically identifying fetal abdominal standard scanning planes and structures, particularly focusing on the abdominal circumference. Ultrasound images from a prospective cohort study were preprocessed using CV2 and Keras-OCR to eliminate textual elements and artifacts. Optical character recognition detected and removed textual components, followed by inpainting using adjacent pixels. Six deep learning neural networks, Xception and MobileNetV3Large, were employed to categorize fetal abdominal view planes. The dataset included nine classes, and the models were evaluated through a tenfold cross-validation cycle. The MobileNet3Large and EfficientV2S achieved accuracy rates of 79.89% and 79.19%, respectively. Data screening confirmed non-normal distribution, but the central limit theorem was applied for statistical analysis. ANOVA test revealed statistically significant differences between the models, while Tukey's post hoc tests showed no difference between MobileNet3Large and EfficientV2S, while outperforming the other networks. AI, specifically MobileNet3Large and EfficientV2S, demonstrated promise in identifying fetal abdominal view planes, showcasing potential benefits for prenatal ultrasound diagnostics. Further studies should compare these AI models with established methods for automatic abdominal circumference measurement to assess overall performance.

Prospective Applications of Artificial Intelligence In Fetal Medicine: A Scoping Review of Recent Updates

Introduction

With the incorporation of artificial intelligence (AI), significant advancements have occurred in the field of fetal medicine, holding the potential to transform prenatal care and diagnostics, promising to revolutionize prenatal care and diagnostics. This scoping review aims to explore the recent updates in the prospective application of AI in fetal medicine, evaluating its current uses, potential benefits, and limitations.

Methods

Compiling literature concerning the utilization of AI in fetal medicine does not appear to modify the subject or provide an exhaustive exploration of electronic databases. Relevant studies, reviews, and articles published in recent years were incorporated to ensure up-to-date data. The selected works were analyzed for common themes, AI methodologies applied, and the scope of AI's integration into fetal medicine practice.

Results

The review identified several key areas where AI applications are making strides in fetal medicine, including prenatal screening, diagnosis of congenital anomalies, and predicting pregnancy complications. AI-driven algorithms have been developed to analyze complex fetal ultrasound data, enhancing image quality and interpretative accuracy. The integration of AI in fetal monitoring has also been explored, with systems designed to identify patterns indicative of fetal distress. Despite these advancements, challenges related to the ethical use of AI, data privacy, and the need for extensive validation of AI tools in diverse populations were noted.

Conclusion

The potential benefits of AI in fetal medicine are immense, offering a brighter future for our field. AI equips us with tools for enhanced diagnosis, monitoring, and prognostic capabilities, promising to revolutionize the way we approach prenatal care and diagnostics. This optimistic outlook underscores the need for further research and interdisciplinary partnerships to fully leverage AI's potential in driving forward the practice of fetal medicine.

Clinical validation of explainable AI for fetal growth scans through multi-level, cross-institutional prospective end-user evaluation

We aimed to develop and evaluate Explainable Artificial Intelligence (XAI) for fetal ultrasound using actionable concepts as feedback to end-users, using a prospective cross-center, multi-level approach. We developed, implemented, and tested a deep-learning model for fetal growth scans using both retrospective and prospective data. We used a modified Progressive Concept Bottleneck Model with pre-established clinical concepts as explanations (feedback on image optimization and presence of anatomical landmarks) as well as segmentations (outlining anatomical landmarks). The model was evaluated prospectively by assessing the following: the model's ability to assess standard plane quality, the correctness of explanations, the clinical usefulness of explanations, and the model's ability to discriminate between different levels of expertise among clinicians. We used 9352 annotated images for model development and 100 videos for prospective evaluation. Overall classification accuracy was 96.3%. The model's performance in assessing standard plane quality was on par with that of clinicians. Agreement between model segmentations and explanations provided by expert clinicians was found in 83.3% and 74.2% of cases, respectively. A panel of clinicians evaluated segmentations as useful in 72.4% of cases and explanations as useful in 75.0% of cases. Finally, the model reliably discriminated between the performances of clinicians with different levels of experience (p- values < 0.01 for all measures) Our study has successfully developed an Explainable AI model for real-time feedback to clinicians performing fetal growth scans. This work contributes to the existing literature by addressing the gap in the clinical validation of Explainable AI models within fetal medicine, emphasizing the importance of multi-level, cross-institutional, and prospective evaluation with clinician end-users. The prospective clinical validation uncovered challenges and opportunities that could not have been anticipated if we had only focused on retrospective development and validation, such as leveraging AI to gauge operator competence in fetal ultrasound.

Deep learning based detection and classification of fetal lip in ultrasound images

OBJECTIVES

Fetal cleft lip is a common congenital defect. Considering the delicacy and difficulty of observing fetal lips, we have utilized deep learning technology to develop a new model aimed at quickly and accurately assessing the development of fetal lips during prenatal examinations. This model can detect ultrasound images of the fetal lips and classify them, aiming to provide a more objective prediction for the development of fetal lips.

METHODS

This study included 632 pregnant women in their mid-pregnancy stage, who underwent ultrasound examinations of the fetal lips, collecting both normal and abnormal fetal lip ultrasound images. To improve the accuracy of the detection and classification of fetal lips, we proposed and validated the Yolov5-ECA model.

RESULTS

The experimental results show that, compared with the currently popular 10 models, our model achieved the best results in the detection and classification of fetal lips. In terms of the detection of fetal lips, the mean average precision (mAP) at 0.5 and mAP at 0.5:0.95 were 0.920 and 0.630, respectively. In the classification of fetal lip ultrasound images, the accuracy reached 0.925.

CONCLUSIONS

The deep learning algorithm has accuracy consistent with manual evaluation in the detection and classification process of fetal lips. This automated recognition technology can provide a powerful tool for inexperienced young doctors, helping them to accurately conduct examinations and diagnoses of fetal lips.

Ensemble learning for fetal ultrasound and maternal-fetal data to predict mode of delivery after labor induction

Providing adequate counseling on mode of delivery after induction of labor (IOL) is of utmost importance. Various AI algorithms have been developed for this purpose, but rely on maternal-fetal data, not including ultrasound (US) imaging. We used retrospectively collected clinical data from 808 subjects submitted to IOL, totaling 2024 US images, to train AI models to predict vaginal delivery (VD) and cesarean section (CS) outcomes after IOL. The best overall model used only clinical data (F1-score: 0.736; positive predictive value (PPV): 0.734). The imaging models employed fetal head, abdomen and femur US images, showing limited discriminative results. The best model used femur images (F1-score: 0.594; PPV: 0.580). Consequently, we constructed ensemble models to test whether US imaging could enhance the clinical data model. The best ensemble model included clinical data and US femur images (F1-score: 0.689; PPV: 0.693), presenting a false positive and false negative interesting trade-off. The model accurately predicted CS on 4 additional cases, despite misclassifying 20 additional VD, resulting in a 6.0% decrease in average accuracy compared to the clinical data model. Hence, integrating US imaging into the latter model can be a new development in assisting mode of delivery counseling.

Cervical elastography: finding a novel predictor for improving the prediction of preterm birth in uncomplicated twin pregnancies

OBJECTIVE

This study set out to investigate a novel ultrasound parameter using cervical elastosonography for improving the prediction of spontaneous preterm birth (sPTB) in twin pregnancies.

STUDY DESIGN

The study was comprised of 106 twin pregnancies from October 2020 to January 2022 in Beijing Obstetrics and Gynecology Hospital. They were divided into two groups according to gestational age (GA) at delivery (delivery < 35 weeks and delivery ≥ 35 weeks). There were five elastographic parameters: Elasticity Contrast Index (ECI), Cervical Hardness Ratio (CHR), Closed Internal cervical ostium Strain rate (CIS); External cervical ostium strain rate (ES), CIS/ES ratio and Cervical Length (CL). All of the clinical and ultrasonic indicators with P < 0.1 were considered candidate indicators via univariate logistic regression. Based on the extracted unified combination of clinical indicators, the combinations of permutation with the candidate ultrasound indicators were performed step by step in multivariable logistic regression. The best ultrasound indicator with the lowest Akaike Information Criterion (AIC) and the highest Areas Under the receiver operating characteristic Curve (AUC) was chosen for establishing the prediction score.

RESULTS

Over 30% (36/106) of those who delivered before 35 weeks gestation. There were distinct differences in the clinical characteristics and cervical elastography parameters between the two groups. Seven major clinical variables were identified as a unified clinical indicator. CISmin as the best ultrasound elastography predictor indicated the lowest AIC and the highest AUC and outperformed alternative indicators significantly in the prediction of delivery before 35 weeks of gestation. Unfortunately, CLmin which was commonly used in clinical practice ranked far from all of the cervical elastography parameters and presented the highest AIC and the lowest AUC. A preliminary scoring rule was established and the ability to predict the risk of sPTB in twin pregnancies was improved (Accuracy: 0.896 vs 0.877; AIC: 81.494 vs 91.698; AUC: 0.923 vs 0.906).

CONCLUSIONS

The cervical elastosonography predictor such as CISmin might be a more useful indicator applied for enhancing the ability in predicting twin pregnancies preterm birth than CL. Furthermore, there would be more benefits for advancing clinical decision-making in actual clinical practice by using cervical elastosonography in the near future.

Discrete placental gene expression signatures accompany diabetic disease classifications during pregnancy

BACKGROUND

Gestational diabetes mellitus affects up to 10% of pregnancies and is classified into subtypes gestational diabetes subtype A1 (GDMA1) (managed by lifestyle modifications) and gestational diabetes subtype A2 (GDMA2) (requiring medication). However, whether these subtypes are distinct clinical entities or more reflective of an extended spectrum of normal pregnancy endocrine physiology remains unclear.

OBJECTIVE

Integrated bulk RNA-sequencing (RNA-seq), single-cell RNA-sequencing (scRNA-seq), and spatial transcriptomics harbors the potential to reveal disease gene signatures in subsets of cells and tissue microenvironments. We aimed to combine these high-resolution technologies with rigorous classification of diabetes subtypes in pregnancy. We hypothesized that differences between preexisting type 2 and gestational diabetes subtypes would be associated with altered gene expression profiles in specific placental cell populations.

STUDY DESIGN

In a large case-cohort design, we compared validated cases of GDMA1, GDMA2, and type 2 diabetes mellitus (T2DM) to healthy controls by bulk RNA-seq (n=54). Quantitative analyses with reverse transcription and quantitative PCR of presumptive genes of significant interest were undertaken in an independent and nonoverlapping validation cohort of similarly well-characterized cases and controls (n=122). Additional integrated analyses of term placental single-cell, single-nuclei, and spatial transcriptomics data enabled us to determine the cellular subpopulations and niches that aligned with the GDMA1, GDMA2, and T2DM gene expression signatures at higher resolution and with greater confidence.

RESULTS

Dimensional reduction of the bulk RNA-seq data revealed that the most common source of placental gene expression variation was the diabetic disease subtype. Relative to controls, we found 2052 unique and significantly differentially expressed genes (-22 thresholds; q<0.05 Wald Test) among GDMA1 placental specimens, 267 among GDMA2, and 1520 among T2DM. Several candidate marker genes (chorionic somatomammotropin hormone 1 [CSH1], period circadian regulator 1 [PER1], phosphatidylinositol-4,5-bisphosphate 3-kinase catalytic subunit beta [PIK3CB], forkhead box O1 [FOXO1], epidermal growth factor receptor [EGFR], interleukin 2 receptor subunit beta [IL2RB], superoxide dismutase 3 [SOD3], dedicator of cytokinesis 5 [DOCK5], suppressor of glucose, and autophagy associated 1 [SOGA1]) were validated in an independent and nonoverlapping validation cohort (q<0.05 Tukey). Functional enrichment revealed the pathways and genes most impacted for each diabetes subtype, and the degree of proximal similarity to other subclassifications. Surprisingly, GDMA1 and T2DM placental signatures were more alike by virtue of increased expression of chromatin remodeling and epigenetic regulation genes, while albumin was the top marker for GDMA2 with increased expression of placental genes in the wound healing pathway. Assessment of these gene signatures in single-cell, single-nuclei, and spatial transcriptomics data revealed high specificity and variability by placental cell and microarchitecture types. For example, at the cellular and spatial (eg, microarchitectural) levels, distinguishing features were observed in extravillous trophoblasts (GDMA1) and macrophages (GDMA2). Lastly, we utilized these data to train and evaluate 4 machine learning models to estimate our confidence in predicting the control or diabetes status of placental transcriptome specimens with no available clinical metadata.

CONCLUSION

Consistent with the distinct association of perinatal outcome risk, placentae from GDMA1, GDMA2, and T2DM-affected pregnancies harbor unique gene signatures that can be further distinguished by altered placental cellular subtypes and microarchitectural niches.

New Horizons of Artificial Intelligence in Medicine and Surgery

Background: Ideas about Artificial intelligence appeared about half a century ago, but only now is it becoming an essential element of everyday life. The data provided are becoming a bigger pool and we need artificial intelligence that will help us with its superhuman powers. Its interaction with medicine is improving more and more, with medicine being a domain that continues to be perfected. Materials and Methods: The most important databases were used to perform this detailed search that addresses artificial intelligence in the medical and surgical fields. Discussion: Machine learning, deep learning, neural networks and computer vision are some of the mechanisms that are becoming a trend in healthcare worldwide. Developed countries such as Japan, France and Germany have already implemented artificial intelligence in their medical systems. The help it gives is in medical diagnosis, patient monitoring, personalized therapy and workflow optimization. Artificial intelligence will help surgeons to perfect their skills, to standardize techniques and to choose the best surgical techniques. Conclusions: The goal is to predict complications, reduce diagnostic times, diagnose complex pathologies, guide surgeons intraoperatively and reduce medical errors. We are at the beginning of this, and the potential is enormous, but we must not forget the impediments that may appear and slow down its implementation.

Detection and Classification of Hysteroscopic Images Using Deep Learning

BACKGROUND

Although hysteroscopy with endometrial biopsy is the gold standard in the diagnosis of endometrial pathology, the gynecologist experience is crucial for a correct diagnosis. Deep learning (DL), as an artificial intelligence method, might help to overcome this limitation. Unfortunately, only preliminary findings are available, with the absence of studies evaluating the performance of DL models in identifying intrauterine lesions and the possible aid related to the inclusion of clinical factors in the model.

AIM

To develop a DL model as an automated tool for detecting and classifying endometrial pathologies from hysteroscopic images.

METHODS

A monocentric observational retrospective cohort study was performed by reviewing clinical records, electronic databases, and stored videos of hysteroscopies from consecutive patients with pathologically confirmed intrauterine lesions at our Center from January 2021 to May 2021. Retrieved hysteroscopic images were used to build a DL model for the classification and identification of intracavitary uterine lesions with or without the aid of clinical factors. Study outcomes were DL model diagnostic metrics in the classification and identification of intracavitary uterine lesions with and without the aid of clinical factors.

RESULTS

We reviewed 1500 images from 266 patients: 186 patients had benign focal lesions, 25 benign diffuse lesions, and 55 preneoplastic/neoplastic lesions. For both the classification and identification tasks, the best performance was achieved with the aid of clinical factors, with an overall precision of 80.11%, recall of 80.11%, specificity of 90.06%, F1 score of 80.11%, and accuracy of 86.74 for the classification task, and overall detection of 85.82%, precision of 93.12%, recall of 91.63%, and an F1 score of 92.37% for the identification task.

CONCLUSION

Our DL model achieved a low diagnostic performance in the detection and classification of intracavitary uterine lesions from hysteroscopic images. Although the best diagnostic performance was obtained with the aid of clinical data, such an improvement was slight.

AI-based preeclampsia detection and prediction with electrocardiogram data

Introduction

More than 76,000 women die yearly from preeclampsia and hypertensive disorders of pregnancy. Early diagnosis and management of preeclampsia can improve outcomes for both mother and baby. In this study, we developed artificial intelligence models to detect and predict preeclampsia from electrocardiograms (ECGs) in point-of-care settings.

Methods

Ten-second 12-lead ECG data was obtained from two large health care settings: University of Tennessee Health Science Center (UTHSC) and Atrium Health Wake Forest Baptist (AHWFB). UTHSC data was split into 80% training and 20% holdout data. The model used a modified ResNet convolutional neural network, taking one-dimensional raw ECG signals comprising 12 channels as an input, to predict risk of preeclampsia. Sub-analyses were performed to assess the predictive accuracy for preeclampsia prediction within 30, 60, or 90 days before diagnosis.

Results

The UTHSC cohort included 904 ECGs from 759 females (78.8% African American) with a mean ± sd age of 27.3 ± 5.0 years. The AHWFB cohort included 817 ECGs from 141 females (45.4 African American) with a mean ± sd age of 27.4 ± 5.9 years. The cross-validated ECG-AI model yielded an AUC (95% CI) of 0.85 (0.77-0.93) on UTHSC holdout data, and an AUC (95% CI) of 0.81 (0.77-0.84) on AHWFB data. The sub-analysis of different time windows before preeclampsia prediction resulted in AUCs (95% CI) of 0.92 (0.84-1.00), 0.89 (0.81-0.98) and 0.90 (0.81-0.98) when tested on ECGs 30 days, 60 days and 90 days, respectively, before diagnosis. When assessed on early onset preeclampsia (preeclampsia diagnosed at <34 weeks of pregnancy), the model's AUC (95% CI) was 0.98 (0.89-1.00).

Discussion

We conclude that preeclampsia can be identified with high accuracy via application of AI models to ECG data.

Predicting preterm birth using auto-ML frameworks: a large observational study using electronic inpatient discharge data

Background

To develop and compare different AutoML frameworks and machine learning models to predict premature birth.

Methods

The study used a large electronic medical record database to include 715,962 participants who had the principal diagnosis code of childbirth. Three Automatic Machine Learning (AutoML) were used to construct machine learning models including tree-based models, ensembled models, and deep neural networks on the training sample (N = 536,971). The area under the curve (AUC) and training times were used to assess the performance of the prediction models, and feature importance was computed via permutation-shuffling.

Results

The H2O AutoML framework had the highest median AUC of 0.846, followed by AutoGluon (median AUC: 0.840) and Auto-sklearn (median AUC: 0.820), and the median training time was the lowest for H2O AutoML (0.14 min), followed by AutoGluon (0.16 min) and Auto-sklearn (4.33 min). Among different types of machine learning models, the Gradient Boosting Machines (GBM) or Extreme Gradient Boosting (XGBoost), stacked ensemble, and random forrest models had better predictive performance, with median AUC scores being 0.846, 0.846, and 0.842, respectively. Important features related to preterm birth included premature rupture of membrane (PROM), incompetent cervix, occupation, and preeclampsia.

Conclusions

Our study highlights the potential of machine learning models in predicting the risk of preterm birth using readily available electronic medical record data, which have significant implications for improving prenatal care and outcomes.

Artificial intelligence as a new answer to old challenges in maternal-fetal medicine and obstetrics

BACKGROUND

Following the latest trends in the development of artificial intelligence (AI), the possibility of processing an immense amount of data has created a breakthrough in the medical field. Practitioners can now utilize AI tools to advance diagnostic protocols and improve patient care.

OBJECTIVE

The aim of this article is to present the importance and modalities of AI in maternal-fetal medicine and obstetrics and its usefulness in daily clinical work and decision-making process.

METHODS

A comprehensive literature review was performed by searching PubMed for articles published from inception up until August 2023, including the search terms "artificial intelligence in obstetrics", "maternal-fetal medicine", and "machine learning" combined through Boolean operators. In addition, references lists of identified articles were further reviewed for inclusion.

RESULTS

According to recent research, AI has demonstrated remarkable potential in improving the accuracy and timeliness of diagnoses in maternal-fetal medicine and obstetrics, e.g., advancing perinatal ultrasound technique, monitoring fetal heart rate during labor, or predicting mode of delivery. The combination of AI and obstetric ultrasound can help optimize fetal ultrasound assessment by reducing examination time and improving diagnostic accuracy while reducing physician workload.

CONCLUSION

The integration of AI in maternal-fetal medicine and obstetrics has the potential to significantly improve patient outcomes, enhance healthcare efficiency, and individualized care plans. As technology evolves, AI algorithms are likely to become even more sophisticated. However, the successful implementation of AI in maternal-fetal medicine and obstetrics needs to address challenges related to interpretability and reliability.

Evolving the Era of 5D Ultrasound? A Systematic Literature Review on the Applications for Artificial Intelligence Ultrasound Imaging in Obstetrics and Gynecology

Artificial intelligence (AI) has gained prominence in medical imaging, particularly in obstetrics and gynecology (OB/GYN), where ultrasound (US) is the preferred method. It is considered cost effective and easily accessible but is time consuming and hindered by the need for specialized training. To overcome these limitations, AI models have been proposed for automated plane acquisition, anatomical measurements, and pathology detection. This study aims to overview recent literature on AI applications in OB/GYN US imaging, highlighting their benefits and limitations. For the methodology, a systematic literature search was performed in the PubMed and Cochrane Library databases. Matching abstracts were screened based on the PICOS (Participants, Intervention or Exposure, Comparison, Outcome, Study type) scheme. Articles with full text copies were distributed to the sections of OB/GYN and their research topics. As a result, this review includes 189 articles published from 1994 to 2023. Among these, 148 focus on obstetrics and 41 on gynecology. AI-assisted US applications span fetal biometry, echocardiography, or neurosonography, as well as the identification of adnexal and breast masses, and assessment of the endometrium and pelvic floor. To conclude, the applications for AI-assisted US in OB/GYN are abundant, especially in the subspecialty of obstetrics. However, while most studies focus on common application fields such as fetal biometry, this review outlines emerging and still experimental fields to promote further research.

AI-Enhanced Analysis Reveals Impact of Maternal Diabetes on Subcutaneous Fat Mass in Fetuses without Growth Alterations

Pregnant women with diabetes often present impaired fetal growth, which is less common if maternal diabetes is well-controlled. However, developing strategies to estimate fetal body composition beyond fetal growth that could better predict metabolic complications later in life is essential. This study aimed to evaluate subcutaneous fat tissue (femur and humerus) in fetuses with normal growth among pregnant women with well-controlled diabetes using a reproducible 3D-ultrasound tool and offline TUI (Tomographic Ultrasound Imaging) analysis. Additionally, three artificial intelligence classifier models were trained and validated to assess the clinical utility of the fetal subcutaneous fat measurement. A significantly larger subcutaneous fat area was found in three-femur and two-humerus selected segments of fetuses from women with diabetes compared to the healthy pregnant control group. The full classifier model that includes subcutaneous fat measure, gestational age, fetal weight, fetal abdominal circumference, maternal body mass index, and fetal weight percentile as variables, showed the best performance, with a detection rate of 70%, considering a false positive rate of 10%, and a positive predictive value of 82%. These findings provide valuable insights into the impact of maternal diabetes on fetal subcutaneous fat tissue as a variable independent of fetal growth.

Evaluation of an artificial intelligent algorithm (Heartassist™) to automatically assess the quality of second trimester cardiac views: a prospective study

OBJECTIVES

The aim of this study was to evaluate the agreement between visual and automatic methods in assessing the adequacy of fetal cardiac views obtained during second trimester ultrasonographic examination.

METHODS

In a prospective observational study frames of the four-chamber view left and right outflow tracts, and three-vessel trachea view were obtained from 120 consecutive singleton low-risk women undergoing second trimester ultrasound at 19-23 weeks of gestation. For each frame, the quality assessment was performed by an expert sonographer and by an artificial intelligence software (Heartassist™). The Cohen's κ coefficient was used to evaluate the agreement rates between both techniques.

RESULTS

The number and percentage of images considered adequate visually by the expert or with Heartassist™ were similar with a percentage >87 % for all the cardiac views considered. The Cohen's κ coefficient values were for the four-chamber view 0.827 (95 % CI 0.662-0.992), 0.814 (95 % CI 0.638-0.990) for left ventricle outflow tract, 0.838 (95 % CI 0.683-0.992) and three vessel trachea view 0.866 (95 % CI 0.717-0.999), indicating a good agreement between the two techniques.

CONCLUSIONS

Heartassist™ allows to obtain the automatic evaluation of fetal cardiac views, reached the same accuracy of expert visual assessment and has the potential to be applied in the evaluation of fetal heart during second trimester ultrasonographic screening of fetal anomalies.

Validation of an automatic software in assessing fetal brain volume from three dimensional ultrasonographic volumes: Comparison with manual analysis

OBJECTIVE

This study was aimed to test the agreement between a manual and an automatic technique in measuring fetal brain volume (FBV) from three-dimensional (3D) fetal head datasets.

METHODS

FBV were acquired independently by two operators from low risk singleton pregnancies at a gestational age between 19 and 34 weeks. FBV measurements were obtained using an automatic software (Smart ICV™) and manually by Virtual Organ Computer-aided AnaLysis (VOCAL™). Intraclass correlation coefficient (ICC) were calculated to assess reliability, while bias and agreement were evaluate by examining Bland-Altman plots. The time spent in measuring volumes was calculated and values obtained compared.

RESULTS

Sixty-three volumes were considered for the study. In all the included volumes successful volume analysis were obtained with both techniques. Smart ICV™ showed a high intra-observer (0.996; 95% CI 0.994-0.998) and inter-observer (ICC 0.995; 95% CI 0.991-0.997). An excellent degree of reliability was found when the two techniques were compared (ICC 0.995; 95% CI 0.987-0.998). The time required to perform FBV was significantly lower for Smart ICV™ than VOCAL™ (8.2 ± 4.5 vs. 121.3 ± 19.0 s; p < 0.0001).

CONCLUSIONS

The measurement of FBV is feasible with both manual and automatic techniques. Smart ICV™ showed an excellent intra- and inter-observer reliability associated with a valuable agreement with volume measurements obtained manually with VOCAL™. Volumes may be measured significantly faster with smart ICV™ than manually and this automatic software has the potential to become the preferred methods for the assessment of FBV.

Evaluating Capabilities of Large Language Models: Performance of GPT4 on Surgical Knowledge Assessments

Background

Artificial intelligence (AI) has the potential to dramatically alter healthcare by enhancing how we diagnosis and treat disease. One promising AI model is ChatGPT, a large general-purpose language model trained by OpenAI. The chat interface has shown robust, human-level performance on several professional and academic benchmarks. We sought to probe its performance and stability over time on surgical case questions.

Methods

We evaluated the performance of ChatGPT-4 on two surgical knowledge assessments: the Surgical Council on Resident Education (SCORE) and a second commonly used knowledge assessment, referred to as Data-B. Questions were entered in two formats: open-ended and multiple choice. ChatGPT output were assessed for accuracy and insights by surgeon evaluators. We categorized reasons for model errors and the stability of performance on repeat encounters.

Results

A total of 167 SCORE and 112 Data-B questions were presented to the ChatGPT interface. ChatGPT correctly answered 71% and 68% of multiple-choice SCORE and Data-B questions, respectively. For both open-ended and multiple-choice questions, approximately two-thirds of ChatGPT responses contained non-obvious insights. Common reasons for inaccurate responses included: inaccurate information in a complex question (n=16, 36.4%); inaccurate information in fact-based question (n=11, 25.0%); and accurate information with circumstantial discrepancy (n=6, 13.6%). Upon repeat query, the answer selected by ChatGPT varied for 36.4% of inaccurate questions; the response accuracy changed for 6/16 questions.

Conclusion

Consistent with prior findings, we demonstrate robust near or above human-level performance of ChatGPT within the surgical domain. Unique to this study, we demonstrate a substantial inconsistency in ChatGPT responses with repeat query. This finding warrants future consideration and presents an opportunity to further train these models to provide safe and consistent responses. Without mental and/or conceptual models, it is unclear whether language models such as ChatGPT would be able to safely assist clinicians in providing care.

Chat Generative Pre-trained Transformer: why we should embrace this technology

With the advent of artificial intelligence that not only can learn from us but also can communicate with us in plain language, humans are embarking on a brave new future. The interaction between humans and artificial intelligence has never been so widespread. Chat Generative Pre-trained Transformer is an artificial intelligence resource that has potential uses in the practice of medicine. As clinicians, we have the opportunity to help guide and develop new ways to use this powerful tool. Optimal use of any tool requires a certain level of comfort. This is best achieved by appreciating its power and limitations. Being part of the process is crucial in maximizing its use in our field. This clinical opinion demonstrates the potential uses of Chat Generative Pre-trained Transformer for obstetrician-gynecologists and encourages readers to serve as the driving force behind this resource.