Validation of an ICD-10-based algorithm to identify stillbirth in the Sentinel System

Development of a Pregnancy Cohort in Commercial Insurance Claims Data: Evaluation of Deliveries Identified From Inpatient Versus Outpatient Claims

PURPOSE

Studies using insurance claims data to identify pregnancies are rarely able to directly assess the validity of the pregnancy/delivery. Inpatient versus outpatient delivery claims may provide different levels of evidence, but more stringent requirements could result in exclusion of true pregnancies. We identified delivery codes from the inpatient and outpatient settings and examined possible confirmatory evidence suggesting that a delivery truly occurred.

METHODS

Using a US commercial insurance database (2006-2021), we identified potential pregnancies by presence of delivery claims from a provider and/or facility. We classified deliveries as inpatient (claim date during inpatient admission) or outpatient (claim date not during inpatient admission). We identified possible confirmatory evidence for each delivery including: (1) Presence of both provider and facility delivery codes; (2) presence of both diagnosis and procedure delivery codes; (3) labor and delivery revenue codes; (4) gestational age diagnosis codes; (5) pregnancy-related care codes; (6) linkage to an infant claim; and (7) infant insurance enrollment and linkage to a birthing parent. We quantified the proportion of deliveries with confirmatory evidence by delivery setting. Among deliveries with ≥ 1 piece of confirmatory evidence, we compared patient characteristics by apparent delivery setting.

RESULTS

Among 4 084 474 delivery episodes, 96.4% were classified as inpatient and 3.6% outpatient. 99.9% of inpatient and 94.0% of outpatient deliveries had ≥ 1 piece of confirmatory evidence. Pregnancy-related care codes were the most common type of confirmatory evidence (99.0% inpatient, 85.7% outpatient). Deliveries classified as inpatient occurred among patients who were older and more clinically complex (i.e., more pregnancy complications, chronic diseases, and prescription medications).

CONCLUSIONS

The vast majority of deliveries had confirmatory evidence regardless of apparent setting. Patient characteristics differed by delivery setting. Inclusion of apparent outpatient deliveries may increase the sample size of the study population and improve the generalizability of study results.

Use of the Sentinel System to Examine Medical Product Use and Outcomes During Pregnancy

While many pregnant individuals use prescription medications, evidence supporting product safety during pregnancy is often inadequate. Existing electronic healthcare data sources provide large, diverse samples of health plan members to allow for the study of medical product utilization during pregnancy, as well as pregnancy, maternal, and infant outcomes. The Sentinel System is a national medical product surveillance system that includes administrative claims and electronic health record databases from large national and regional health insurers. In addition to these data sources, Sentinel develops and maintains a sizeable selection of analytic tools to facilitate epidemiologic analyses in a way that protects patient privacy and health system autonomy. In this article, we provide an overview of Sentinel System infrastructure, including the Mother-Infant Linkage Table, parameterizable analytic tools, and algorithms to estimate gestational age and identify pregnancy outcomes. We also describe past and future Sentinel work that contributes to our understanding of the way medical products are used and the safety of these products during pregnancy.

Internal validation of gestational age estimation algorithms in health-care databases using pregnancies conceived through fertility procedures

Fertility procedures recorded in health-care databases can be used to estimate the start of pregnancy, which can serve as a reference standard to validate gestational age estimates based on International Classification of Diseases codes. In a cohort of 17 398 US MarketScan pregnancies (2011-2020) in which conception was achieved via fertility procedures, we estimated gestational age at the end of pregnancy using algorithms based on (1) time (days) since the fertility procedure (the reference standard); (2) International Classification of Diseases, Ninth Revision (ICD-9)/International Classification of Diseases, Tenth Revision (ICD-10) (before/after October 2015) codes indicating gestational length recorded at the end of pregnancy (method A); and (3) ICD-10 end-of-pregnancy codes enhanced with Z3A codes denoting specific gestation weeks recorded at prenatal visits (method B). We calculated the proportion of pregnancies with an estimated gestational age falling within 14 days ($\pm$14 days) of the reference standard. Method A accuracy was similar for ICD-9 and ICD-10 codes. After 2015, method B was more accurate than method A: For term births, within-14-day agreement was 90.8% for method A and 98.7% for method B. Corresponding estimates were 70.1% and 95.6% for preterm births; 35.3% and 92.6% for stillbirths; 54.3% and 64.2% for spontaneous abortions; and 16.7% and 84.6% for elective terminations. ICD-10-based algorithms that incorporate Z3A codes improve the accuracy of gestational age estimation in health-care databases, especially for preterm births and non-live births.

Drug Exposure During Pregnancy: A Case-Control Study from a Primary Care Database

Objective

Drug exposure during pregnancy is frequent, even more during first trimester as pregnant women might not be aware of their condition. We used available electronic health records (EHRs) to describe the use of medications during the first trimester in pregnant women and to compare drug exposure between those women who had an abortion (either elective or spontaneous) compared to those who had live births.

Materials and Methods

Case-control study of abortions, either elective or spontaneous (cases), and live birth pregnancies (controls) in Sistema d'Informació per al Desenvolupament de la Investigació en Atenció Primària (Catalan Primary Health electronic health records) from 2012 to 2020. Exposure to drugs during first trimester of pregnancy was considered to estimate the association with abortion by conditional logistic regression and adjusted by health conditions and other drugs exposure.

Results

Sixty thousand three hundred fifty episodes of abortions were matched to 118,085 live birth pregnancy episodes. Cases had higher rates of alcohol intake (9.9% vs. 7.2%, p < 0.001), smoking (4.5% vs. 3.6%, p < 0.001), and previous abortions (9.9% vs. 7.8%, p < 0.001). Anxiety (30.3% and 25.1%, p < 0.001), respiratory diseases (10.6% and 9.2%, p < 0.001), and migraine (8.2% and 7.3%, p < 0.001), for cases and controls, respectively, were the most frequent baseline conditions. Cases had lower rate of drug exposure, 40,148 (66.5%) versus 80,449 (68.1%), p < 0.001. Association with abortion was found for systemic antihistamines (adjusted odds ratio [ORadj] 1.23, 95% confidence interval [CI] 1.19-1.27), antidepressants (ORadj 1.11, 95% CI 1.06-1.17), anxiolytics (ORadj 1.31, 95% CI 1.26-1.73), and nonsteroidal anti-inflammatory drugs (ORadj 1. 63, 95% CI 1.59-1.67).

Conclusions

These high rates of drug exposures during the first trimester of pregnancy highlights the relevance of informed prescription to women with childbearing potential.

Six Years of the US Food and Drug Administration's Postmarket Active Risk Identification and Analysis System in the Sentinel Initiative: Implications for Real World Evidence Generation

Congress mandated the creation of a postmarket Active Risk Identification and Analysis (ARIA) system containing data on 100 million individuals for monitoring risks associated with drug and biologic products using data from disparate sources to complement the US Food and Drug Administration's (FDA's) existing postmarket capabilities. We report on the first 6 years of ARIA utilization in the Sentinel System (2016-2021). The FDA has used the ARIA system to evaluate 133 safety concerns; 54 of these evaluations have closed with regulatory determinations, whereas the rest remain in progress. If the ARIA system and the FDA's Adverse Event Reporting System are deemed insufficient to address a safety concern, then the FDA may issue a postmarket requirement to a product's manufacturer. One hundred ninety-seven ARIA insufficiency determinations have been made. The most common situation for which ARIA was found to be insufficient is the evaluation of adverse pregnancy and fetal outcomes following in utero drug exposure, followed by neoplasms and death. ARIA was most likely to be sufficient for thromboembolic events, which have high positive predictive value in claims data alone and do not require supplemental clinical data. The lessons learned from this experience illustrate the continued challenges using administrative claims data, especially to define novel clinical outcomes. This analysis can help to identify where more granular clinical data are needed to fill gaps to improve the use of real-world data for drug safety analyses and provide insights into what is needed to efficiently generate high-quality real-world evidence for efficacy.

Validation of diagnosis codes in healthcare databases in Taiwan, a literature review

PURPOSE

To compile validation findings of diagnosis codes and related algorithms for health outcomes of interest from National Health Insurance (NHI) or electronic medical records in Taiwan.

METHODS

We carried out a literature review of English articles in PubMed® and Embase from 2000 through July 2022 with appropriate search terms. Potentially relevant articles were identified through review of article titles and abstracts, full text search of methodology terms "validation", "positive predictive value", and "algorithm" in Subjects & Methods (or Methods) and Results sections of articles, followed by full text review of potentially eligible articles.

RESULTS

We identified 50 published reports with validation findings of diagnosis codes and related algorithms for a wide range of health outcomes of interest in Taiwan, including cardiovascular diseases, stroke, renal impairment, malignancy, diabetes, mental health diseases, respiratory diseases, viral (B and C) hepatitis, and tuberculosis. Most of the reported PPVs were in the 80% ~ 99% range. Assessment of algorithms based on ICD-10 systems were reported in 8 articles, all published in 2020 or later.

CONCLUSIONS

Investigators have published validation reports that may serve as empirical evidence to evaluate the utility of secondary health data environment in Taiwan for research and regulatory purpose.

Validation of an ICD-9-Based Algorithm to Identify Stillbirth Episodes from Medicaid Claims Data

INTRODUCTION

In administrative data, accurate timing of exposure relative to gestation is critical for determining the effect of potential teratogen exposure on pregnancy outcomes.

OBJECTIVE

To develop an algorithm for identifying stillbirth episodes in the ICD-9-CM era using national Medicaid claims data (1999-2014).

METHODS

Unique stillbirth episodes were identified from clusters of medical claims using a hierarchy that identified the encounter with the highest potential of including the actual stillbirth delivery and that delineated subsequent pregnancy episodes. Each episode was validated using clinical detail on retrieved medical records as the gold standard.

RESULTS

Among 220 retrieved records, 197 were usable for validation of 1417 stillbirth episodes identified by the algorithm. The positive predictive value (PPV) was 64.0% (57.3-70.7%) overall, 80.4% (73.8-87.1%) for inpatient episodes, 28.2% (14.1-42.3%) for outpatient-only episodes, and 20.0% (2.5-37.5%) for outpatient episodes with overlapping hospitalizations. The absolute difference between the dates of the algorithm-specified stillbirth delivery and the medical record-based event was 4.2 ± 24.3 days overall, 1.7 ± 7.7 days for inpatient episodes, 14.3 ± 51.4 days for outpatient-only episodes, and 1.0 ± 2.0 days for outpatient episodes that overlapped with a hospitalization. Excluding all outpatient episodes, as well as pregnancies involving multiple births, the PPV increased to 82.7% (76.8-89.8%).

CONCLUSIONS

Our algorithm to identify stillbirths from administrative claims data had a moderately high PPV. Positive predictive value was substantially increased by restricting the setting to inpatient episodes and using only input diagnostic codes for singleton stillbirths.

Validation of claims-based algorithms to identify non-live birth outcomes

PURPOSE

Perinatal epidemiology studies using healthcare utilization databases are often restricted to live births, largely due to the lack of established algorithms to identify non-live births. The study objective was to develop and validate claims-based algorithms for the ascertainment of non-live births.

METHODS

Using the Mass General Brigham Research Patient Data Registry 2000-2014, we assembled a cohort of women enrolled in Medicaid with a non-live birth. Based on ≥1 inpatient or ≥2 outpatient diagnosis/procedure codes, we identified and randomly sampled 100 potential stillbirth, spontaneous abortion, and termination cases each. For the secondary definitions, we excluded cases with codes for other pregnancy outcomes within ±5 days of the outcome of interest and relaxed the definitions for spontaneous abortion and termination by allowing cases with one outpatient diagnosis only. Cases were adjudicated based on medical chart review. We estimated the positive predictive value (PPV) for each outcome.

RESULTS

The PPV was 71.0% (95% CI, 61.1-79.6) for stillbirth; 79.0% (69.7-86.5) for spontaneous abortion, and 93.0% (86.1-97.1) for termination. When excluding cases with adjacent codes for other pregnancy outcomes and further relaxing the definition, the PPV increased to 80.6% (69.5-88.9) for stillbirth, 86.6% (80.5-91.3) for spontaneous abortion and 94.9% (91.1-97.4) for termination. The PPV for the composite outcome using the relaxed definition was 94.4% (92.3-96.1).

CONCLUSIONS

Our findings suggest non-live birth outcomes can be identified in a valid manner in epidemiological studies based on healthcare utilization databases.

Development and Validation of ICD-10-CM-based Algorithms for Date of Last Menstrual Period, Pregnancy Outcomes, and Infant Outcomes

INTRODUCTION AND OBJECTIVE

Validation studies of algorithms for pregnancy outcomes based on International Classification of Diseases, 10th Revision, Clinical Modification (ICD-10-CM) codes are important for conducting drug safety research using administrative claims databases. To facilitate the conduct of pregnancy safety studies, this exploratory study aimed to develop and validate ICD-10-CM-based claims algorithms for date of last menstrual period (LMP) and pregnancy outcomes using medical records.

METHODS

Using a mother-infant-linked claims database, the study included women with a pregnancy between 2016-2017 and their infants. Claims-based algorithms for LMP date utilized codes for gestational age (Z3A codes). The primary outcomes were major congenital malformations (MCMs) and spontaneous abortion; additional secondary outcomes were also evaluated. Each pregnancy outcome was identified using a claims-based simple algorithm, defined as presence of ≥ 1 claim for the outcome. Positive predictive values (PPV) and 95% confidence intervals (CI) were calculated.

RESULTS

Overall, 586 medical records were sought and 365 (62.3%) were adjudicated, including 125 records each for MCMs and spontaneous abortion. Last menstrual period date was validated among maternal charts procured for pregnancy outcomes and fewer charts were adjudicated for the secondary outcomes. The median difference in days between LMP date based on Z3A codes and adjudicated LMP date was 4.0 (interquartile range: 2.0-10.0). The PPV of the simple algorithm for spontaneous abortion was 84.7% (95% CI 78.3, 91.2). The PPV for the MCM algorithm was < 70%. The algorithms for the secondary outcomes pre-eclampsia, premature delivery, and low birthweight performed well, with PPVs > 70%.

CONCLUSIONS

The ICD-10-CM claims-based algorithm for spontaneous abortion performed well and may be used in pregnancy studies. Further algorithm refinement for MCMs is needed. The algorithms for LMP date and the secondary outcomes would benefit from additional validation in a larger sample.

Development and validation of claims-based algorithms to identify pregnancy based on data from a university hospital in Japan

OBJECTIVE

When using administrative data, validation is essential since these data are not collected for research purposes and misclassification can occur. Thus, this study aimed to develop algorithms identifying pregnancy and to evaluate the validity of administrative claims data in Japan.

METHODS

All females who visited the Tohoku University Hospital Department of Obstetrics in 2018 were included. The diagnosis, medical procedure, medication, and medical service addition fee data were utilized to identify pregnancy, with the electronic medical records set as the gold standard. Combination algorithms were developed using predefined pregnancy-related claims data with a positive predictive value (PPV) ≥80%. Sensitivity (SE), specificity (SP), PPV, and negative predictive value (NPV) with their corresponding 95% confidence intervals (CIs) were calculated for these combination algorithms.

RESULTS

This study included 1757 females with a mean age of 32.8 (standard deviation: 5.9) years. In general, the individual claims data were able to identify pregnancy with a PPV ≥80%; however, the number of pregnancies identified using a single claims data was limited. Based on the combination algorithm with all of the categories, including diagnosis, medical procedure, medication, and medical service addition, the calculated SE, SP, PPV, and NPV were 73.4% (95% CI: 71.2%-75.4%), 96.9% (95% CI: 89.3%-99.6%), 99.8%,(95% CI: 99.4%-100.0%), and 12.3% (95% CI: 9.6%-15.4%), respectively.

CONCLUSIONS

The combination algorithm to identify pregnancy demonstrated a high PPV and moderate SE. The algorithm validated in this study is expected to accelerate future studies that aim to identify pregnancies and evaluate pregnancy outcome.

Identification of pregnancies and infants within a United States commercial healthcare administrative claims database

PURPOSE

Health care insurance claims databases are becoming a more common data source for studies of medication safety during pregnancy. While pregnancies have historically been identified in such databases by pregnancy outcomes, International Classification of Diseases, 10^th revision Clinical Modification (ICD-10-CM) Z3A codes denoting weeks of gestation provide more granular information on pregnancies and pregnancy periods (i.e. start and end dates). The purpose of this study was to develop a process that uses Z3A codes to identify pregnancies, pregnancy periods, and links infants within a commercial health insurance claims database.

METHODS

We identified pregnancies, gestation periods, pregnancy outcomes, and linked infants within the United States (US)-based Optum Research Database (ORD) between 2015 and 2020 via a series of algorithms utilizing diagnosis and procedure codes on claims. The diagnosis and procedure codes included ICD-10-CM codes, Current Procedural Terminology (CPT) codes, and Healthcare Common Procedure Coding System (HCPCS) codes.

RESULTS

We identified 1,030,874 pregnancies among 841,196 women of reproductive age. Of pregnancies with livebirth outcomes, 84% were successfully linked to infants. The prevalence of pregnancy outcomes (livebirth, stillbirth, ectopic, molar, abortion) was similar to national estimates.

CONCLUSIONS

This process provides an opportunity to study drug safety and care patterns during pregnancy and may be replicated in other claims databases containing ICD-10-CM, CPT, and HCPCS codes. Work is underway to validate and refine the various algorithms. This article is protected by copyright. All rights reserved.

Validity of Administrative Data for Identifying Birth-Related Outcomes with the End Date of Pregnancy in a Japanese University Hospital

This study aimed to develop and validate claims-based algorithms for identifying live birth, fetal death, and cesarean section by utilizing administrative data from a university hospital in Japan. We included women who visited the Department of Obstetrics at a university hospital in 2018. The diagnosis, medical procedures, and medication data were used to identify potential cases of live birth, fetal death, and cesarean section. By reviewing electronic medical records, we evaluated the positive predictive values (PPVs) and the accuracy of the end date of pregnancy for each claims datum. “Selected algorithm 1” based on PPVs and “selected algorithm 2” based on both the PPVs and the accuracy of the end date of pregnancy were developed. A total of 1757 women were included, and the mean age was 32.8 years. The PPVs of “selected algorithm 1” and “selected algorithm 2” were both 98.1% for live birth, 99.0% and 98.9% for fetal death, and 99.7% and 100.0% for cesarean section, respectively. These findings suggest that the developed algorithms are useful for future studies for evaluating live birth, fetal death, and cesarean section with an accurate end date of pregnancy.

Utility of fertility procedures and prenatal tests to estimate gestational age for live-births and stillbirths in electronic health plan databases

PURPOSE

Current algorithms to evaluate gestational age (GA) during pregnancy rely on hospital coding at delivery and are not applicable to non-live births. We developed an algorithm using fertility procedures and fertility tests, without relying on delivery coding, to develop a novel GA algorithm in live-births and stillbirths.

METHODS

Three pregnancy cohorts were identified from 16 health-plans in the Sentinel System: 1) hospital admissions for live-birth, 2) hospital admissions for stillbirth, and 3) medical chart-confirmed stillbirths. Fertility procedures and prenatal tests, recommended within specific GA windows were evaluated for inclusion in our GA algorithm. Our GA algorithm was developed against a validated delivery-based GA algorithm in live-births, implemented within a sample of chart-confirmed stillbirths, and compared to national estimates of GA at stillbirth.

RESULTS

Our algorithm, including fertility procedures and 11 prenatal tests, assigned a GA at delivery to 97.9% of live-births and 92.6% of stillbirths. For live-births (n = 4 701 207), it estimated GA within two weeks of a reference delivery-based GA algorithm in 82.5% of pregnancies, with a mean difference of 3.7 days. In chart-confirmed stillbirths (n = 49), it estimated GA within two weeks of the clinically recorded GA at delivery for 80% of pregnancies, with a mean difference of 11.1 days. Implementation of the algorithm in a cohort of stillbirths (n = 40 484) had an increased percentage of deliveries after 36 weeks compared to national estimates.

CONCLUSIONS

In a population of primarily commercially-insured pregnant women, fertility procedures and prenatal tests can estimate GA with sufficient sensitivity and accuracy for utility in pregnancy studies.