OUP accepted manuscript

Development of Lung Cancer Risk Prediction Machine Learning Models for Equitable Learning Health System: Retrospective Study

BACKGROUND

A significant proportion of young at-risk patients and nonsmokers are excluded by the current guidelines for lung cancer (LC) screening, resulting in low-screening adoption. The vision of the US National Academy of Medicine to transform health systems into learning health systems (LHS) holds promise for bringing necessary structural changes to health care, thereby addressing the exclusivity and adoption issues of LC screening.

OBJECTIVE

This study aims to realize the LHS vision by designing an equitable, machine learning (ML)-enabled LHS unit for LC screening. It focuses on developing an inclusive and practical LC risk prediction model, suitable for initializing the ML-enabled LHS (ML-LHS) unit. This model aims to empower primary physicians in a clinical research network, linking central hospitals and rural clinics, to routinely deliver risk-based screening for enhancing LC early detection in broader populations.

METHODS

We created a standardized data set of health factors from 1397 patients with LC and 1448 control patients, all aged 30 years and older, including both smokers and nonsmokers, from a hospital's electronic medical record system. Initially, a data-centric ML approach was used to create inclusive ML models for risk prediction from all available health factors. Subsequently, a quantitative distribution of LC health factors was used in feature engineering to refine the models into a more practical model with fewer variables.

RESULTS

The initial inclusive 250-variable XGBoost model for LC risk prediction achieved performance metrics of 0.86 recall, 0.90 precision, and 0.89 accuracy. Post feature refinement, a practical 29-variable XGBoost model was developed, displaying performance metrics of 0.80 recall, 0.82 precision, and 0.82 accuracy. This model met the criteria for initializing the ML-LHS unit for risk-based, inclusive LC screening within clinical research networks.

CONCLUSIONS

This study designed an innovative ML-LHS unit for a clinical research network, aiming to sustainably provide inclusive LC screening to all at-risk populations. It developed an inclusive and practical XGBoost model from hospital electronic medical record data, capable of initializing such an ML-LHS unit for community and rural clinics. The anticipated deployment of this ML-LHS unit is expected to significantly improve LC-screening rates and early detection among broader populations, including those typically overlooked by existing screening guidelines.

Review of Interventions That Improve Uptake of Lung Cancer Screening: A Cataloging of Strategies That Have Been Shown to Work (or Not)

TOPIC IMPORTANCE

Lung cancer screening (LCS) has the potential to decrease mortality from lung cancer by 20%. Yet, more than a decade since LCS was established as an evidence-based practice, < 20% of the eligible population in the United States has been screened. This review focuses on critically appraising interventions that have been designed to increase the initial uptake of LCS, including how they address known barriers to LCS and their effectiveness in overcoming these barriers.

REVIEW FINDINGS

Studies were categorized based on the primary barriers that they addressed: (1) identifying eligible patients (including enhancing awareness through smoking history collection, outreach, and education), (2) shared decision-making-related interventions, and (3) patient navigation interventions. Four of the studies included multicomponent interventions, which often included patient navigation as one of the components. Overall, the effectiveness of the studies reviewed at improving LCS uptake generally was modest and was limited by the multilevel barriers that need to be overcome. Multicomponent interventions generally were more effective at improving LCS uptake, but most studies still had relatively low completion of screening.

SUMMARY

Improving uptake of LCS requires learning from prior interventions to design multilevel interventions that address barriers to LCS at key steps and identifying which components of these interventions are effective and generalizable.

Implementation of an Electronic Medical Record Alert Significantly Increases Lung Cancer Screening Uptake

INTRODUCTION

Lung cancer survival is significantly improved with early detection. However, lung cancer screening (LCS) uptake remains low despite national recommendations. Our aim was to determine whether implementation of an electronic medical record (EMR) alert and order set would increase LCS uptake.

STUDY DESIGN

A query of current and former smokers identified 62,630 patients aged 50 and above in the primary care setting between January 1, 2021 and May 5, 2022. We randomly reviewed 3704 charts for LCS eligibility and recorded who received LCS in the form of low-dose computed tomography amongst the eligible patients. We collected demographic information including gender, race, primary language, ethnicity, zip code, and insurance. Data analysis was performed utilizing 2-proportional z tests.

RESULTS

We identified 461 patients who were LCS eligible. Our overall LCS uptake was 19.9% (92/461). Three-time frames were analyzed: (1) prior to EMR alert implementation, (2) after implementation of EMR alert (January 7, 2021), and (3) after implementation of EMR alert and order set (March 3, 2021). Screening uptake was significantly improved with initiation of EMR alert (1/46 [2.2%] to 23/109 [21.1%]; P = .003). LCS uptake remained similarly high after subsequent order set implementation (23/109 [21.1%] and 68/306 [22.2%]; P = .72). Amongst the different demographics, age was significantly associated with screening uptake, with age ≥65 demonstrating statistically significant increased rates of screening (15.6% [41/263] for <65 vs 25.8% [51/198] for ≥65; P = .007).

CONCLUSION

Implementation of EMR alerts significantly improves LCS uptake in the primary care setting. Such efforts should be considered in other hospital settings to improve LCS uptake.

Concordance Between Electronic Health Record-Recorded Race and Ethnicity and Patient Report in Emergency Department Patients

OBJECTIVE

We assessed the concordance of patient-reported race and ethnicity for emergency department (ED) patients compared with what was recorded in the electronic health record.

METHODS

We conducted a single-center, prospective, observational study of 744 ED patients (English- and/or Spanish-speaking), asking them to describe their race and ethnicity. We compared the distributions of ethnicity and race between patient-reported and electronic health record data using McNemar's test. We calculated percent agreement and Cohen's kappa, with 95% confidence intervals (CI), for the concordance of patient-reported race and ethnicity with electronic health record data.

RESULTS

Of 744 ED patients, 731 participants who completed the survey reported their ethnicity, resulting in 98.2% of electronic health records obtained ethnicities matched self-reported data (kappa = 0.95; 95% CI: 0.92 to 0.98). For those who self-reported as Hispanic, only 92.3% agreement was observed between the self-reported and electronic health record values. For all patients who had race recorded, 85.4% agreement was observed (kappa = 0.75; 95% CI 0.71 to 0.79). High rates of agreement were observed for Black or African American patients (98.7%) and White patients (96.6%), with low rates for those who identified as "More than one race" (22.9%) or "Other" race (1.8%). In the subset of Hispanic patients, low rates of agreement (25.0%) were observed for race (kappa = 0.10; 95% CI 0.01 to 0.19).

CONCLUSIONS

Documentation discordance regarding race and ethnicity exists between electronic health records and self-reported data for our ED patients, particularly for ethnically Hispanic and Latino/a patients. Future efforts should focus on ensuring that demographic information in the electronic health record is accurately collected.

Early Detection and Interception of Lung Cancer

Recent advances in lung cancer treatment have led to dramatic improvements in 5-year survival rates. And yet, lung cancer remains the leading cause of cancer-related mortality, in large part, because it is often diagnosed at an advanced stage, when cure is no longer possible. Lung cancer screening (LCS) is essential for intercepting the disease at an earlier stage. Unfortunately, LCS has been poorly adopted in the United States, with less than 5% of eligible patients being screened nationally. This article will describe the data supporting LCS, the obstacles to LCS implementation, and the promising opportunities that lie ahead.

Melatonin and Risk of Age-Related Macular Degeneration

Importance

Melatonin has been shown to oppose several processes that are known to mediate age-related macular degeneration (AMD), but whether melatonin can confer benefits against AMD remains unclear.

Objective

To examine the association between melatonin supplementation and the risk of the development or progression of AMD.

Design, Setting, and Participants

This retrospective cohort study accessed data from TriNetX, a national database of deidentified electronic medical records from both inpatient and outpatient health care organizations across the US, between December 4, 2023, and March 19, 2024. Patients aged 50 years or older, 60 years or older, and 70 years or older with no history of AMD (AMD-naive group) and with a history of nonexudative AMD (nonexudative AMD group) were queried for instances of melatonin medication codes between November 14, 2008, and November 14, 2023. Patients were then classified into either a melatonin group or a control group based on the presence of medication codes for melatonin. Propensity score matching (PSM) was performed to match the cohorts based on demographic variables, comorbidities, and nonmelatonin hypnotic medication use.

Exposure

The presence of at least 4 instances of melatonin records that each occurred at least 3 months apart.

Main Outcomes and Measures

After PSM, the melatonin and the control cohorts were compared to evaluate the risk ratios (RRs) and the 95% CIs of having an outcome. For the AMD-naive group, the outcome was defined as a new diagnosis of any AMD, whereas for the nonexudative AMD group, the outcome was progression to exudative AMD.

Results

Among 121 523 patients in the melatonin-naive group aged 50 years or older (4848 in the melatonin cohort [4580 after PSM; mean (SD) age, 68.24 (11.47) years; 2588 female (56.5%)] and 116 675 in the control cohort [4580 after PSM; mean (SD) age, 68.17 (10.63) years; 2681 female (58.5%)]), melatonin use was associated with a reduced risk of developing AMD (RR, 0.42; 95% CI, 0.28-0.62). Among 66 253 patients aged 50 years or older in the nonexudative AMD group (4350 in the melatonin cohort [4064 after PSM; mean (SD) age, 80.21 (8.78) years; 2482 female (61.1%)] and 61 903 in the control cohort [4064 patients after PSM; mean (SD) age, 80.31 (8.03) years; 2531 female (62.3%)]), melatonin was associated with a reduced risk of AMD progression to exudative AMD (RR, 0.44; 95% CI, 0.34-0.56). The results were consistent among subsets of individuals aged 60 years or older (AMD-naive cohort: RR, 0.36 [95% CI, 0.25-0.54]; nonexudative AMD cohort: RR, 0.38 [95% CI, 0.30-0.49]) and 70 years or older (AMD-naive cohort: RR, 0.35 [95% CI, 0.23-0.53]; nonexudative AMD cohort: RR, 0.40 [95% CI, 0.31-0.51]).

Conclusions and Relevance

Melatonin use was associated with a decreased risk of development and progression of AMD. Although lifestyle factors may have influenced this association, these findings provide a rationale for further research on the efficacy of using melatonin as a preventive therapy against AMD.

Pack-Year Smoking History: An Inadequate and Biased Measure to Determine Lung Cancer Screening Eligibility

PURPOSE

Pack-year smoking history is an imperfect and biased measure of cumulative tobacco exposure. The use of pack-year smoking history to determine lung cancer screening eligibility in the current US Preventive Services Task Force (USPSTF) guideline may unintentionally exclude many high-risk individuals, especially those from racial and ethnic minority groups. It is unclear whether using a smoking duration cutoff instead of a smoking pack-year cutoff would improve the selection of individuals for screening.

METHODS

We analyzed 49,703 individuals with a smoking history from the Southern Community Cohort Study (SCCS) and 22,126 individuals with a smoking history from the Black Women's Health Study (BWHS) to assess eligibility for screening under the USPSTF guideline versus a proposed guideline that replaces the ≥20-pack-year criterion with a ≥20-year smoking duration criterion.

RESULTS

Under the USPSTF guideline, only 57.6% of Black patients with lung cancer in the SCCS would have qualified for screening, whereas a significantly higher percentage of White patients with lung cancer (74.0%) would have qualified (P < .001). Under the proposed guideline, the percentage of Black and White patients with lung cancer who would have qualified for screening increased to 85.3% and 82.0%, respectively, eradicating the disparity in screening eligibility between the groups. In the BWHS, using a 20-year smoking duration cutoff instead of a 20-pack-year cutoff increased the percentage of Black women with lung cancer who would have qualified for screening from 42.5% to 63.8%.

CONCLUSION

Use of a 20-year smoking duration cutoff instead of a 20-pack-year cutoff greatly increases the proportion of patients with lung cancer who would qualify for screening and eliminates the racial disparity in screening eligibility between Black versus White individuals; smoking duration has the added benefit of being easier to calculate and being a more precise assessment of smoking exposure compared with pack-year smoking history.

Lung Cancer Screening Before and After a Multifaceted Electronic Health Record Intervention: A Nonrandomized Controlled Trial

Importance

Lung cancer is the deadliest cancer in the US. Early-stage lung cancer detection with lung cancer screening (LCS) through low-dose computed tomography (LDCT) improves outcomes.

Objective

To assess the association of a multifaceted clinical decision support intervention with rates of identification and completion of recommended LCS-related services.

Design, Setting, and Participants

This nonrandomized controlled trial used an interrupted time series design, including 3 study periods from August 24, 2019, to April 27, 2022: baseline (12 months), period 1 (11 months), and period 2 (9 months). Outcome changes were reported as shifts in the outcome level at the beginning of each period and changes in monthly trend (ie, slope). The study was conducted at primary care and pulmonary clinics at a health care system headquartered in Salt Lake City, Utah, among patients aged 55 to 80 years who had smoked 30 pack-years or more and were current smokers or had quit smoking in the past 15 years. Data were analyzed from September 2023 through February 2024.

Interventions

Interventions in period 1 included clinician-facing preventive care reminders, an electronic health record-integrated shared decision-making tool, and narrative LCS guidance provided in the LDCT ordering screen. Interventions in period 2 included the same clinician-facing interventions and patient-facing reminders for LCS discussion and LCS.

Main Outcome and Measure

The primary outcome was LCS care gap closure, defined as the identification and completion of recommended care services. LCS care gap closure could be achieved through LDCT completion, other chest CT completion, or LCS shared decision-making.

Results

The study included 1865 patients (median [IQR] age, 64 [60-70] years; 759 female [40.7%]). The clinician-facing intervention (period 1) was not associated with changes in level but was associated with an increase in slope of 2.6 percentage points (95% CI, 2.4-2.7 percentage points) per month in care gap closure through any means and 1.6 percentage points (95% CI, 1.4-1.8 percentage points) per month in closure through LDCT. In period 2, introduction of patient-facing reminders was associated with an immediate increase in care gap closure (2.3 percentage points; 95% CI, 1.0-3.6 percentage points) and closure through LDCT (2.4 percentage points; 95% CI, 0.9-3.9 percentage points) but was not associated with an increase in slope. The overall care gap closure rate was 175 of 1104 patients (15.9%) at the end of the baseline period vs 588 of 1255 patients (46.9%) at the end of period 2.

Conclusions and Relevance

In this study, a multifaceted intervention was associated with an improvement in LCS care gap closure.

Trial Registration

ClinicalTrials.gov Identifier: NCT04498052.

Overcoming barriers to lung cancer screening using a systemwide approach with additional focus on the non-screened

BACKGROUND

The lung cancer screening program at St Elizabeth Healthcare (Kentucky, USA) began in 2013. Over 33,000 low-dose computed tomography lung cancer screens have been performed. From 2015 through 2021, 2595 lung cancers were diagnosed systemwide. A Screening Program with Impactful Results from Early Detection, reviews that experience; 342 (13.2%) were diagnosed by screening and 2253 (86.8%) were non-screened. As a secondary objective, the non-screened cohort was queried to determine how many additional individuals could have been screened, identifying barriers and failures to meet eligibility.

METHODS

Our QlikSense database extracted the lung cancer patients from the Cancer Patient Data and Management System, and identified and categorized them separately as screened or non-screened populations. Stage distribution was compared in screened and non-screened groups. Those meeting age criteria, with any smoking history, were further queried for screening eligibility, accessing the electronic medical record smoking history and audit trail, and determining if enough information was available to substantiate screening eligibility. The same methodology was applied to CMS 2015 and USPSTF 2021 criteria.

RESULTS

The screened and non-screened patients were accounted for in a stage migration chart demonstrating clear shift to early stage among screened lung cancer patients. Additionally, analysis of non-screened individuals is presented.

CONCLUSION

Of the St Elizabeth Healthcare eligible patients attributed to primary care providers, 49.6% were screened in 2021. Despite this level of success, this study highlighted a sizeable pool of additional individuals that could have been screened. We are shifting focus to the non-screened pool of patients that meet eligibility, further enhancing the impact on our community.

Protocol to measure validity and reliability of colorectal, breast, cervical and lung cancer screening questions from the 2021 National Health Interview Survey: Methodology and design

Previous studies demonstrate that self-reports of mammography screening for breast cancer and colonoscopy screening for colorectal cancer demonstrate concordance, based on adherence to screening guidelines, with electronic medical records (EMRs) in over 90% of those interviewed, as well as high sensitivity and specificity, and can be used for monitoring our Healthy People goals. However, for screening tests for cervical and lung cancers, and for various sub-populations, concordance between self-report and EMRs has been noticeably lower with poor sensitivity or specificity. This study aims to test the validity and reliability of lung, colorectal, cervical, and breast cancer screening questions from the 2021 and 2022 National Health Interview Survey (NHIS). We present the protocol for a study designed to measure the validity and reliability of the NHIS cancer screening questions compared to EMRs from four US-based healthcare systems. We planned a randomized trial of a phone- vs web-based survey with NHIS questions that were previously revised based on extensive cognitive interviewing. Our planned sample size will be 1576 validity interviews, and 1260 interviews randomly assigned at 1 or 3 months after the initial interview. We are enrolling people eligible for cancer screening based on age, sex, and smoking history per US Preventive Services Task Force recommendations. We will evaluate question validity using concordance, sensitivity, specificity, positive predictive value, negative predictive value, and report-to-records ratio. We further are randomizing participants to complete a second survey 1 vs 3 months later to assess question reliability. We suggest that typical measures of concordance may need to be reconsidered in evaluating cancer screening questions.

Efficacy of Digital Outreach Strategies for Collecting Smoking Data: Pragmatic Randomized Trial

BACKGROUND

Tobacco smoking is an important risk factor for disease, but inaccurate smoking history data in the electronic medical record (EMR) limits the reach of lung cancer screening (LCS) and tobacco cessation interventions. Patient-generated health data is a novel approach to documenting smoking history; however, the comparative effectiveness of different approaches is unclear.

OBJECTIVE

We designed a quality improvement intervention to evaluate the effectiveness of portal questionnaires compared to SMS text message-based surveys, to compare message frames, and to evaluate the completeness of patient-generated smoking histories.

METHODS

We randomly assigned patients aged between 50 and 80 years with a history of tobacco use who identified English as a preferred language and have never undergone LCS to receive an EMR portal questionnaire or a text survey. The portal questionnaire used a "helpfulness" message, while the text survey tested frame types informed by behavior economics ("gain," "loss," and "helpfulness") and nudge messaging. The primary outcome was the response rate for each modality and framing type. Completeness and consistency with documented structured smoking data were also evaluated.

RESULTS

Participants were more likely to respond to the text survey (191/1000, 19.1%) compared to the portal questionnaire (35/504, 6.9%). Across all text survey rounds, patients were less responsive to the "helpfulness" frame compared with the "gain" frame (odds ratio [OR] 0.29, 95% CI 0.09-0.91; P<.05) and "loss" frame (OR 0.32, 95% CI 11.8-99.4; P<.05). Compared to the structured data in the EMR, the patient-generated data were significantly more likely to be complete enough to determine LCS eligibility both compared to the portal questionnaire (OR 34.2, 95% CI 3.8-11.1; P<.05) and to the text survey (OR 6.8, 95% CI 3.8-11.1; P<.05).

CONCLUSIONS

We found that an approach using patient-generated data is a feasible way to engage patients and collect complete smoking histories. Patients are likely to respond to a text survey using "gain" or "loss" framing to report detailed smoking histories. Optimizing an SMS text message approach to collect medical information has implications for preventative and follow-up clinical care beyond smoking histories, LCS, and smoking cessation therapy.

The SIMPLE Architectural Pattern for Integrating Patient-Facing Apps into Clinical Workflows: Desiderata and Application for Lung Cancer Screening

In December 2022, regulations from the U.S. Office of the National Coordinator for Health IT came into effect that require electronic health record (EHR) systems to accept the connection of any patient-facing digital health app using the SMART on FHIR standard. However, little has been reported with regard to architectural patterns that can be reused to take advantage of this industry development and integrate patient-facing apps into clinical workflows. To address this need, we propose SIMPLE, short for Standards-based Implementation Maximizing Portability Leveraging the EHR. The SIMPLE architectural pattern was designed to meet several key desiderata: do not require patients to install new software; do not retain patient data outside of the EHR; leverage EHRs' existing personal health record (PHR) capabilities to optimize user experience; and maximize portability. Using this pattern, an application for lung cancer screening known as MyLungHealth has been designed and is undergoing iterative user-centered enhancement.

Use of Epic Electronic Health Record System for Health Care Research: Scoping Review

BACKGROUND

Electronic health records (EHRs) enable health data exchange across interconnected systems from varied settings. Epic is among the 5 leading EHR providers and is the most adopted EHR system across the globe. Despite its global reach, there is a gap in the literature detailing how EHR systems such as Epic have been used for health care research.

OBJECTIVE

The objective of this scoping review is to synthesize the available literature on use cases of the Epic EHR for research in various areas of clinical and health sciences.

METHODS

We used established scoping review methods and searched 9 major information repositories, including databases and gray literature sources. To categorize the research data, we developed detailed criteria for 5 major research domains to present the results.

RESULTS

We present a comprehensive picture of the method types in 5 research domains. A total of 4669 articles were screened by 2 independent reviewers at each stage, while 206 articles were abstracted. Most studies were from the United States, with a sharp increase in volume from the year 2015 onwards. Most articles focused on clinical care, health services research and clinical decision support. Among research designs, most studies used longitudinal designs, followed by interventional studies implemented at single sites in adult populations. Important facilitators and barriers to the use of Epic and EHRs in general were identified. Important lessons to the use of Epic and other EHRs for research purposes were also synthesized.

CONCLUSIONS

The Epic EHR provides a wide variety of functions that are helpful toward research in several domains, including clinical and population health, quality improvement, and the development of clinical decision support tools. As Epic is reported to be the most globally adopted EHR, researchers can take advantage of its various system features, including pooled data, integration of modules and developing decision support tools. Such research opportunities afforded by the system can contribute to improving quality of care, building health system efficiencies, and conducting population-level studies. Although this review is limited to the Epic EHR system, the larger lessons are generalizable to other EHRs.

Access to Lung Cancer Screening

Rural and racial/ethnic minority communities experience higher risk and mortality from lung cancer. Lung cancer screening with low-dose computed tomography reduces mortality. However, disparities persist in the uptake of lung cancer screening, especially in marginalized communities. Barriers to lung cancer screening are multilevel and include patient, provider, and system-level barriers. This discussion highlights the key barriers faced by rural and racial/ethnic minority communities.

Electronic Health Record Prompt to Improve Lung Cancer Screening in Primary Care

INTRODUCTION

Lung cancer is the leading cause of cancer death in the U.S. Combusted tobacco use, the primary risk factor, accounts for 90% of all lung cancers. Early detection of lung cancer improves survival, yet lung cancer screening rates are much lower than those of other cancer screening tests. Electronic health record (EHR) systems are an underutilized tool that could improve screening rates.

METHODS

This study was conducted in the Rutgers Robert Wood Johnson Medical Group, a university-affiliated network in New Brunswick, NJ. Two novel EHR workflow prompts were implemented on July 1, 2018. These prompts included fields to determine tobacco use and lung cancer screening eligibility and facilitated low-dose computed tomography ordering for eligible patients. The prompts were designed to improve tobacco use data entry, allowing for better lung cancer screening eligibility identification. Data were analyzed in 2022 retrospectively for the period July 1, 2017 to June 30, 2019. The analyses represented 48,704 total patient visits.

RESULTS

The adjusted odds of patient record completeness to determine eligibility for low-dose computed tomography (AOR=1.19, 95% CI=1.15, 1.23), eligibility for low-dose computed tomography (AOR=1.59, 95% CI=1.38, 1.82), and whether low-dose computed tomography was ordered (AOR=1.04, 95% CI=1.01, 1.07) all significantly increased after the electronic medical record prompts were implemented.

CONCLUSIONS

These findings show the utility and benefit of EHR prompts in primary care settings to increase identification for lung cancer screening eligibility as well as increased low-dose computed tomography ordering.

Implementation of Lung Cancer Screening in Primary Care and Pulmonary Clinics: Pragmatic Clinical Trial of Electronic Health Record-Integrated Everyday Shared Decision-Making Tool and Clinician-Facing Prompts

BACKGROUND

Although low-dose CT (LDCT) scan imaging lung cancer screening (LCS) can reduce lung cancer mortality, it remains underused. Shared decision-making (SDM) is recommended to assess the balance of benefits and harms for each patient.

RESEARCH QUESTION

Do clinician-facing electronic health record (EHR) prompts and an EHR-integrated everyday SDM tool designed to support routine incorporation of SDM into primary care improve LDCT scan imaging ordering and completion?

STUDY DESIGN AND METHODS

A preintervention and postintervention analysis was conducted in 30 primary care and four pulmonary clinics for visits with patients who met United States Preventive Services Task Force criteria for LCS. Propensity scores were used to adjust for covariates. Subgroup analyses were conducted based on the expected benefit from screening (high benefit vs intermediate benefit), pulmonologist involvement (ie, whether the patient was seen in a pulmonary clinic in addition to a primary care clinic), sex, and race and ethnicity.

RESULTS

In the 12-month preintervention phase among 1,090 eligible patients, 77 patients (7.1%) had LDCT scan imaging orders and 48 patients (4.4%) completed screenings. In the 9-month intervention phase among 1,026 eligible patients, 280 patients (27.3%) had LDCT scan imaging orders and 182 patients (17.7%) completed screenings. Adjusted ORs were 4.9 (95% CI, 3.4-6.9; P < .001) and 4.7 (95% CI, 3.1-7.1; P < .001) for LDCT imaging ordering and completion, respectively. Subgroup analyses showed increases in ordering and completion for all patient subgroups. In the intervention phase, the SDM tool was used by 23 of 102 ordering providers (22.5%) and for 69 of 274 patients (25.2%) for whom LDCT scan imaging was ordered and who needed SDM at the time of ordering.

INTERPRETATION

Clinician-facing EHR prompts and an EHR-integrated everyday SDM tool are promising approaches to improving LCS in the primary care setting. However, room for improvement remains. As such, further research is warranted.

TRIAL REGISTRY

ClinicalTrials.gov; No.: NCT04498052; URL: www.

CLINICALTRIALS

gov.

Considerations for using predictive models that include race as an input variable: The case study of lung cancer screening

Indiscriminate use of predictive models incorporating race can reinforce biases present in source data and lead to an exacerbation of health disparities. In some countries, such as the United States, there is therefore a push to remove race from prediction models; however, there are still many prediction models that use race as an input. Biomedical informaticists who are given the responsibility of using these predictive models in healthcare environments are likely to be faced with questions like how to deal with race covariates in these models. Thus, there is a need for a pragmatic framework to help model users think through how to include race in their chosen model so as to avoid inadvertently exacerbating disparities. In this paper, we use the case study of lung cancer screening to propose a simple framework to guide how model users can approach the use (or non-use) of race inputs in the predictive models they are tasked with leveraging in electronic health records and clinical workflows.

The accuracy of race & ethnicity data in US based healthcare databases: A systematic review

BACKGROUND

The availability and accuracy of data on a patient's race/ethnicity varies across databases. Discrepancies in data quality can negatively impact attempts to study health disparities.

METHODS

We conducted a systematic review to organize information on the accuracy of race/ethnicity data stratified by database type and by specific race/ethnicity categories.

RESULTS

The review included 43 studies. Disease registries showed consistently high levels of data completeness and accuracy. EHRs frequently showed incomplete and/or inaccurate data on the race/ethnicity of patients. Databases had high levels of accurate data for White and Black patients but relatively high levels of misclassification and incomplete data for Hispanic/Latinx patients. Asians, Pacific Islanders, and AI/ANs are the most misclassified. Systems-based interventions to increase self-reported data showed improvement in data quality.

CONCLUSION

Data on race/ethnicity that is collected with the purpose of research and quality improvement appears most reliable. Data accuracy can vary by race/ethnicity status and better collection standards are needed.

Development and Multicenter Case-Control Validation of Urinary Comprehensive Genomic Profiling for Urothelial Carcinoma Diagnosis, Surveillance, and Risk-Prediction

PURPOSE

Urinary comprehensive genomic profiling (uCGP) uses next-generation sequencing to identify mutations associated with urothelial carcinoma and has the potential to improve patient outcomes by noninvasively diagnosing disease, predicting grade and stage, and estimating recurrence risk.

EXPERIMENTAL DESIGN

This is a multicenter case-control study using banked urine specimens collected from patients undergoing initial diagnosis/hematuria workup or urothelial carcinoma surveillance. A total of 581 samples were analyzed by uCGP: 333 for disease classification and grading algorithm development, and 248 for blinded validation. uCGP testing was done using the UroAmp platform, which identifies five classes of mutation: single-nucleotide variants, copy-number variants, small insertion-deletions, copy-neutral loss of heterozygosity, and aneuploidy. UroAmp algorithms predicting urothelial carcinoma tumor presence, grade, and recurrence risk were compared with cytology, cystoscopy, and pathology.

RESULTS

uCGP algorithms had a validation sensitivity/specificity of 95%/90% for initial cancer diagnosis in patients with hematuria and demonstrated a negative predictive value (NPV) of 99%. A positive diagnostic likelihood ratio (DLR) of 9.2 and a negative DLR of 0.05 demonstrate the ability to risk-stratify patients presenting with hematuria. In surveillance patients, binary urothelial carcinoma classification demonstrated an NPV of 91%. uCGP recurrence-risk prediction significantly prognosticated future recurrence (hazard ratio, 6.2), whereas clinical risk factors did not. uCGP demonstrated positive predictive value (PPV) comparable with cytology (45% vs. 42%) with much higher sensitivity (79% vs. 25%). Finally, molecular grade predictions had a PPV of 88% and a specificity of 95%.

CONCLUSIONS

uCGP enables noninvasive, accurate urothelial carcinoma diagnosis and risk stratification in both hematuria and urothelial carcinoma surveillance patients.

Electronic Health Record-Based Algorithms as Part of the Solution for Improving Lung Cancer Screening

New EHR-based algorithm in #ClinicalCancerInformatics offers a glimpse into the future of lung cancer screening eligibility prediction. Great promise, yet hurdles in implementation and comprehensive strategies for screening are needed for a substantial impact.

Development of an Electronic Health Record-Based Algorithm for Predicting Lung Cancer Screening Eligibility in the Population-Based Research to Optimize the Screening Process Lung Research Consortium

PURPOSE

Lung cancer screening (LCS) guidelines in the United States recommend LCS for those age 50-80 years with at least 20 pack-years smoking history who currently smoke or quit within the last 15 years. We tested the performance of simple smoking-related criteria derived from electronic health record (EHR) data and developed and tested the performance of a multivariable model in predicting LCS eligibility.

METHODS

Analyses were completed within the Population-based Research to Optimize the Screening Process Lung Consortium (PROSPR-Lung). In our primary validity analyses, the reference standard LCS eligibility was based on self-reported smoking data collected via survey. Within one PROSPR-Lung health system, we used a training data set and penalized multivariable logistic regression using the Least Absolute Shrinkage and Selection Operator to select EHR-based variables into the prediction model including demographics, smoking history, diagnoses, and prescription medications. A separate test data set assessed model performance. We also conducted external validation analysis in a separate health system and reported AUC, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and accuracy metrics associated with the Youden Index.

RESULTS

There were 14,214 individuals with survey data to assess LCS eligibility in primary analyses. The overall performance for assigning LCS eligibility status as measured by the AUC values at the two health systems was 0.940 and 0.938. At the Youden Index cutoff value, performance metrics were as follows: accuracy, 0.855 and 0.895; sensitivity, 0.886 and 0.920; specificity, 0.896 and 0.850; PPV, 0.357 and 0.444; and NPV, 0.988 and 0.992.

CONCLUSION

Our results suggest that health systems can use an EHR-derived multivariable prediction model to aid in the identification of those who may be eligible for LCS.

Estimating Pack-Year Eligibility for Lung Cancer Screening Using 2 Yes or No Questions

This cross-sectional study describes the development and testing the accuracy of using 2 yes or no questions to estimate pack-year eligibility for lung cancer screening.

Use of Veterans Health Administration Structured Data to Identify Patients Eligible for Lung Cancer Screening

INTRODUCTION

Lung cancer screening (LCS) uptake is low. Assessing patients' cigarette pack-years and years since quitting is challenging given the lack of documentation in structured electronic health record data.

MATERIALS AND METHODS

We used a convenience sample of patients with a chest CT scan in the Veterans Health Administration. We abstracted data on cigarette use from electronic health record notes to determine LCS eligibility based on the 2021 U.S. Preventive Services Task Force age and cigarette use eligibility criteria. We used these data as the "ground truth" of LCS eligibility to compare them with structured data regarding tobacco use and a COPD diagnosis. We calculated sensitivity and specificity as well as fast-and-frugal decision trees.

RESULTS

For 50-80-year-old veterans identified as former or current tobacco users, we obtained 94% sensitivity and 47% specificity. For 50-80-year-old veterans identified as current tobacco users, we obtained 59% sensitivity and 79% specificity. Our fast-and-frugal decision tree that included a COPD diagnosis had a sensitivity of 69% and a specificity of 60%.

CONCLUSION

These results can help health care systems make their LCS outreach efforts more efficient and give administrators and researchers a simple method to estimate their number of possibly eligible patients.

Enhancing an enterprise data warehouse for research with data extracted using natural language processing

Objective

This study aims to develop a generalizable architecture for enhancing an enterprise data warehouse for research (EDW4R) with results from a natural language processing (NLP) model, which allows discrete data derived from clinical notes to be made broadly available for research use without need for NLP expertise. The study also quantifies the additional value that information extracted from clinical narratives brings to EDW4R.

Materials and methods

Clinical notes written during one month at an academic health center were used to evaluate the performance of an existing NLP model and to quantify its value added to the structured data. Manual review was utilized for performance analysis. The architecture for enhancing the EDW4R is described in detail to enable reproducibility.

Results

Two weeks were needed to enhance EDW4R with data from 250 million clinical notes. NLP generated 16 and 39% increase in data availability for two variables.

Discussion

Our architecture is highly generalizable to a new NLP model. The positive predictive value obtained by an independent team showed only slightly lower NLP performance than the values reported by the NLP developers. The NLP showed significant value added to data already available in structured format.

Conclusion

Given the value added by data extracted using NLP, it is important to enhance EDW4R with these data to enable research teams without NLP expertise to benefit from value added by NLP models.

A Closer Look-Who Are We Screening for Lung Cancer?

Objective

To evaluate the characteristics of individuals receiving lung cancer screening (LCS) and identify those with potentially limited benefit owing to coexisting chronic illnesses and/or comorbidities.

Patients and Methods

In this retrospective study in the United States, patients were selected from a large clinical database who received LCS from January 1, 2019, through December 31, 2019, with at least 1 year of continuous enrollment. We assessed for potentially limited benefit in LCS defined strictly as not meeting the traditional risk factor inclusion criteria (age <55 years or >80 years, previous computed tomography scan within 11 months before an LCS examination, or a history of nonskin cancer) or liberally as having the potential exclusion criteria related to comorbid life-limiting conditions, such as cardiac and/or respiratory disease.

Results

A total of 51,551 patients were analyzed. Overall, 8391 (16.3%) individuals experienced a potentially limited benefit from LCS. Among those who did not meet the strict traditional inclusion criteria, 317 (3.8%) were because of age, 2350 (28%) reported a history of nonskin malignancy, and 2211 (26.3%) underwent a previous computed tomography thorax within 11 months before an LCS examination. Of those with potentially limited benefit owing to comorbidity, 3680 (43.9%) were because of severe respiratory comorbidity (937 [25.5%] with any hospitalization for coronary obstructive pulmonary disease, interstitial lung disease, or respiratory failure; 131 [3.6%] with hospitalization for respiratory failure requiring mechanical ventilation; or 3197 [86.9%] with chronic obstructive disease/interstitial lung disease requiring outpatient oxygen) and 721 (8.59%) with cardiac comorbidity.

Conclusion

Up to 1 of 6 low-dose computed tomography examinations may have limited benefit from LCS.

Meaningless Use: Assessing Compliance With a Clinically Meaningless Emergency Department Documentation Requirement

Introduction A New York State initiative requests that Emergency Department (ED) providers document in the electronic health record (EHR) each admitted patient's employment status and, if applicable, their mode of commute. This initiative diverts them from their primary duties and increases the likelihood they will either disregard the request or input incorrect information to complete the data fields as fast as possible. This study intends to understand how well providers adhere to this regulation, which, while important for society as a whole, has little clinical relevance, especially in the ED, where the focus is to identify and treat emergent conditions. We hypothesized that clinician-collected employment data would contain many more "N/A" responses than registration-collected employment data (the "gold standard").  Methods We took a randomly selected convenience sample of 100 patients admitted from the ED and compared each patient's provider-entered response to the employment data field to the registration-recorded response. The EHR operates such that the "Employment" field must be completed in order to complete the admission electronically. Data fields collected were: last name, first name, date of birth, medical record number, date and time of arrival, date and time of admission, attending physician, resident physician (if there was one), mid-level provider (if there was one), provider-entered employment status, registration-entered employment status, admitting service (eg, Medicine, Surgery, OB/Gyn), and disposition level (eg, ICU). We assessed the percent of employment data that was concordant between the provider's entry and the registration clerk's entry. We also assessed for the potential confounding variable of how busy the ED was at time of admission, as providers may not take ask about employment or enter such data during particularly busy times. Finally, we interviewed providers to elicit reasons they did not enter accurate data. Statistical significance was set a priori at p <0.05. Results One hundred six patients were screened; six were excluded because one of the authors (MR) was their attending physician. For 92 of the remaining 100 patients, providers recorded employment as "N/A," and for eight patients they recorded "retired." For seven of these eight patients, provider entry matched registration entry (87.5% concordance). To adjust for whether how busy the ED was may have impacted the accuracy of data entry, admissions were categorized according to what time of day the patient was admitted. There was no statistically significant correlation between how busy the ED was and accuracy of data entry. The majority of providers stated they responded "NA" because the employment information was unrelated to the ED visit.  Conclusion In New York, for each patient admitted from the ED, the ED provider is requested to enter the patient's job information and, if they commute to work, the method they use. However, this takes providers' attention away from what they should be doing most: diagnosing and treating patients. This study highlights the unintended consequence of requesting data fields that are not clinically relevant and, from the patient and provider perspective, are not good investments of time and energy and distract from the clinical visit. Persons interpreting such clinically irrelevant data should do so with caution, as the results are unlikely to reflect the truth of what the questions intend to determine.

Impact of the COVID-19 Pandemic on Women's Health Care Access: A Cross-Sectional Study

Background: There has been limited study of how the COVID-19 pandemic has affected women's health care access. Our study aims to examine the prevalence and correlates of COVID-19-related disruptions to (1) primary care; (2) gynecologic care; and (3) preventive health care among women. Materials and Methods: We recruited 4,000 participants from a probability-based online panel. We conducted four multinomial logistic regression models, one for each of the study outcomes: (1) primary care access; (2) gynecologic care access; (3) patient-initiated disruptions to preventive visits; and (4) provider-initiated disruptions to preventive visits. Results: The sample included 1,285 women. One in four women (28.5%) reported that the pandemic affected their primary care access. Sexual minority women (SMW) (odds ratios [OR]: 1.67; 95% confidence intervals [CI]: 1.19-2.33) had higher odds of reporting pandemic-related effects on primary care access compared to women identifying as heterosexual. Cancer survivors (OR: 2.07; 95% CI: 1.25-3.42) had higher odds of reporting pandemic-related effects on primary care access compared to women without a cancer history. About 16% of women reported that the pandemic affected their gynecologic care access. Women with a cancer history (OR: 2.34; 95% CI: 1.35-4.08) had higher odds of reporting pandemic-related effects on gynecologic care compared to women without a cancer history. SMW were more likely to report patient- and provider-initiated delays in preventive health care. Other factors that affected health care access included income, insurance status, and having a usual source of care. Conclusions: The COVID-19 pandemic disrupted women's health care access and disproportionately affected access among SMW and women with a cancer history, suggesting that targeted interventions may be needed to ensure adequate health care access during the COVID-19 pandemic.

Validation of a Deep Learning-Based Model to Predict Lung Cancer Risk Using Chest Radiographs and Electronic Medical Record Data

IMPORTANCE

Lung cancer screening with chest computed tomography (CT) prevents lung cancer death; however, fewer than 5% of eligible Americans are screened. CXR-LC, an open-source deep learning tool that estimates lung cancer risk from existing chest radiograph images and commonly available electronic medical record (EMR) data, may enable automated identification of high-risk patients as a step toward improving lung cancer screening participation.

OBJECTIVE

To validate CXR-LC using EMR data to identify individuals at high-risk for lung cancer to complement 2022 US Centers for Medicare & Medicaid Services (CMS) lung cancer screening eligibility guidelines.

DESIGN, SETTING, AND PARTICIPANTS

This prognostic study compared CXR-LC estimates with CMS screening guidelines using patient data from a large US hospital system. Included participants were persons who currently or formerly smoked cigarettes with an outpatient posterior-anterior chest radiograph between January 1, 2013, and December 31, 2014, with no history of lung cancer or screening CT. Data analysis was performed between May 2021 and June 2022.

EXPOSURES

CXR-LC lung cancer screening eligibility (previously defined as having a 3.297% or greater 12-year risk) based on inputs (chest radiograph image, age, sex, and whether currently smoking) extracted from the EMR.

MAIN OUTCOMES AND MEASURES

6-year incident lung cancer.

RESULTS

A total of 14 737 persons were included in the study population (mean [SD] age, 62.6 [6.8] years; 7154 [48.5%] male; 204 [1.4%] Asian, 1051 [7.3%] Black, 432 [2.9%] Hispanic, 12 330 [85.2%] White) with a 2.4% rate of incident lung cancer over 6 years (361 patients with cancer). CMS eligibility could be determined in 6277 patients (42.6%) using smoking pack-year and quit-date from the EMR. Patients eligible by both CXR-LC and 2022 CMS criteria had a high rate of lung cancer (83 of 974 patients [8.5%]), higher than those eligible by 2022 CMS criteria alone (5 of 177 patients [2.8%]; P < .001). Patients eligible by CXR-LC but not 2022 CMS criteria also had a high 6-year incidence of lung cancer (121 of 3703 [3.3%]). In the 8460 cases (57.4%) where CMS eligibility was unknown, CXR-LC eligible patients had a 5-fold higher rate of lung cancer than ineligible (127 of 5177 [2.5%] vs 18 of 2283 [0.5%]; P < .001). Similar results were found in subgroups, including female patients and Black persons.

CONCLUSIONS AND RELEVANCE

Using routine chest radiographs and other data automatically extracted from the EMR, CXR-LC identified high-risk individuals who may benefit from lung cancer screening CT.

Evaluation of Cancer Deaths Attributable to Tobacco in California, 2014-2019

IMPORTANCE

California's tobacco control efforts have been associated with a decrease in cancer mortality, but these estimates are based on smoking prevalence of the general population. Patient-level tobacco use information allows for more precise estimates of the proportion of cancer deaths attributable to smoking.

OBJECTIVE

To calculate the proportion (smoking-attributable fraction) and number (smoking-attributable cancer mortality) of cancer deaths attributable to tobacco use using patient-level data.

DESIGN, SETTING, AND PARTICIPANTS

The smoking-attributable fraction and smoking-attributable cancer mortality were calculated for a retrospective cohort of patients whose cancer was diagnosed from 2014 to 2019 with at least 1 year of follow-up using relative risks from large US prospective studies and patient-level smoking information. Follow-up continued through April 2022. A population-based cohort was identified from the California Cancer Registry. Participants included adults aged 20 years and older with a diagnosis of 1 of the 12 tobacco-related cancers (oral cavity or pharynx, larynx, esophagus, lung, liver, stomach, pancreas, kidney, bladder, colon or rectum, cervix, and acute myeloid leukemia).

EXPOSURES

Tobacco use defined as current, former, or never.

MAIN OUTCOMES AND MEASURES

The primary outcomes were the smoking-attributable fraction and smoking-attributable cancer mortality for each of the 12 tobacco-related cancers over 2 time periods (2014-2016 vs 2017-2019) and by sex.

RESULTS

Among 395 459 patients with a tobacco-related cancer, most (285 768 patients [72.3%]) were older than 60 years, the majority (228 054 patients [57.7%]) were non-Hispanic White, 229 188 patients were men (58.0%), and nearly one-half (184 415 patients [46.6%]) had lung or colorectal cancers. Nearly one-half of the deaths (93 764 patients [45.8%]) in the cohort were attributable to tobacco. More than one-half (227 660 patients [57.6%]) of patients had ever used tobacco, and 69 103 patients (17.5%) were current tobacco users, which was higher than the proportion in the general population (11.7%). The overall smoking-attributable fraction of cancer deaths decreased significantly from 47.7% (95% CI, 47.3%-48.0%) in 2014 to 2016 to 44.8% (95% CI, 44.5%-45.1%) in 2017 to 2019, and this decrease was seen for both men and women. The overall smoking-attributable cancer mortality decreased by 10.2%.

CONCLUSIONS AND RELEVANCE

California still has a substantial burden of tobacco use and associated cancer. The proportion of cancer deaths associated with tobacco use was almost double what was previously estimated. There was a modest but significant decline in this proportion for overall tobacco-associated cancers, especially for women.

Dissecting the clinicopathologic, genomic, and immunophenotypic correlates of KRASG12D-mutated non-small-cell lung cancer

BACKGROUND

Allele-specific KRAS inhibitors are an emerging class of cancer therapies. KRAS-mutant (KRASMUT) non-small-cell lung cancers (NSCLCs) exhibit heterogeneous outcomes, driven by differences in underlying biology shaped by co-mutations. In contrast to KRASG12C NSCLC, KRASG12D NSCLC is associated with low/never-smoking status and is largely uncharacterized.

PATIENTS AND METHODS

Clinicopathologic and genomic information were collected from patients with NSCLCs harboring a KRAS mutation at the Dana-Farber Cancer Institute (DFCI), Memorial Sloan Kettering Cancer Center, MD Anderson Cancer Center, and Imperial College of London. Multiplexed immunofluorescence for CK7, programmed cell death protein 1 (PD-1), programmed death-ligand 1 (PD-L1), Foxp3, and CD8 was carried out on a subset of samples with available tissue at the DFCI. Clinical outcomes to PD-(L)1 inhibition ± chemotherapy were analyzed according to KRAS mutation subtype.

RESULTS

Of 2327 patients with KRAS-mutated (KRASMUT) NSCLC, 15% (n = 354) harbored KRASG12D. Compared to KRASnon-G12D NSCLC, KRASG12D NSCLC had a lower pack-year (py) smoking history (median 22.5 py versus 30.0 py, P < 0.0001) and was enriched in never smokers (22% versus 5%, P < 0.0001). KRASG12D had lower PD-L1 tumor proportion score (TPS) (median 1% versus 5%, P < 0.01) and lower tumor mutation burden (TMB) compared to KRASnon-G12D (median 8.4 versus 9.9 mt/Mb, P < 0.0001). Of the samples which underwent multiplexed immunofluorescence, KRASG12D had lower intratumoral and total CD8+PD1+ T cells (P < 0.05). Among 850 patients with advanced KRASMUT NSCLC who received PD-(L)1-based therapies, KRASG12D was associated with a worse objective response rate (ORR) (15.8% versus 28.4%, P = 0.03), progression-free survival (PFS) [hazard ratio (HR) 1.51, 95% confidence interval (CI) 1.45-2.00, P = 0.003], and overall survival (OS; HR 1.45, 1.05-1.99, P = 0.02) to PD-(L)1 inhibition alone but not to chemo-immunotherapy combinations [ORR 30.6% versus 35.7%, P = 0.51; PFS HR 1.28 (95%CI 0.92-1.77), P = 0.13; OS HR 1.36 (95%CI 0.95-1.96), P = 0.09] compared to KRASnon-G12D.

CONCLUSIONS

KRASG12D lung cancers harbor distinct clinical, genomic, and immunologic features compared to other KRAS-mutated lung cancers and worse outcomes to PD-(L)1 blockade. Drug development for KRASG12D lung cancers will have to take these differences into account.

Dense phenotyping from electronic health records enables machine learning-based prediction of preterm birth

BACKGROUND

Identifying pregnancies at risk for preterm birth, one of the leading causes of worldwide infant mortality, has the potential to improve prenatal care. However, we lack broadly applicable methods to accurately predict preterm birth risk. The dense longitudinal information present in electronic health records (EHRs) is enabling scalable and cost-efficient risk modeling of many diseases, but EHR resources have been largely untapped in the study of pregnancy.

METHODS

Here, we apply machine learning to diverse data from EHRs with 35,282 deliveries to predict singleton preterm birth.

RESULTS

We find that machine learning models based on billing codes alone can predict preterm birth risk at various gestational ages (e.g., ROC-AUC = 0.75, PR-AUC = 0.40 at 28 weeks of gestation) and outperform comparable models trained using known risk factors (e.g., ROC-AUC = 0.65, PR-AUC = 0.25 at 28 weeks). Examining the patterns learned by the model reveals it stratifies deliveries into interpretable groups, including high-risk preterm birth subtypes enriched for distinct comorbidities. Our machine learning approach also predicts preterm birth subtypes (spontaneous vs. indicated), mode of delivery, and recurrent preterm birth. Finally, we demonstrate the portability of our approach by showing that the prediction models maintain their accuracy on a large, independent cohort (5978 deliveries) from a different healthcare system.

CONCLUSIONS

By leveraging rich phenotypic and genetic features derived from EHRs, we suggest that machine learning algorithms have great potential to improve medical care during pregnancy. However, further work is needed before these models can be applied in clinical settings.