Re-CHARGE-AF: Recalibration of the CHARGE-AF Model for Atrial Fibrillation Risk Prediction in Patients With Acute Stroke

Predicting post-stroke cognitive impairment using electronic health record data

BACKGROUND

Secondary prevention interventions to reduce post-stroke cognitive impairment (PSCI) can be aided by the early identification of high-risk individuals who would benefit from risk factor modification.

AIMS

To develop and evaluate a predictive model to identify patients at increased risk of PSCI over 5 years using data easily accessible from electronic health records.

METHODS

Cohort study that included primary care patients from two academic medical centers. Patients were aged 45 years or older, without prior stroke or prevalent cognitive impairment, with primary care visits and an incident ischemic stroke between 2003 and 2016 (development/internal validation cohort) or 2010 and 2022 (external validation cohort). Predictors of PSCI were ascertained from the electronic health record. The outcome was incident dementia/cognitive impairment within 5 years and beginning 3 months following stroke, ascertained using International Classification of Diseases, Ninth/Tenth Revision (ICD-9/10) codes. For model variable selection, we considered potential predictors of PSCI and constructed 400 bootstrap samples with two-thirds of the model derivation sample. We ran 10-fold cross-validated Cox proportional hazards models using a least absolute shrinkage and selection operator (LASSO) penalty. Variables selected in >25% of samples were included.

RESULTS

The analysis included 332 incident diagnoses of PSCI in the development cohort (n = 3741), and 161 and 128 incident diagnoses in the internal (n = 1925) and external (n = 2237) validation cohorts, respectively. The C-statistic for predicting PSCI was 0.731 (95% confidence interval (CI): 0.694-0.768) in the internal validation cohort, and 0.724 (95% CI: 0.681-0.766) in the external validation cohort. A risk score based on the beta coefficients of predictors from the development cohort stratified patients into low (0-7 points), intermediate (8-11 points), and high (12-23 points) risk groups. The hazard ratios (HRs) for incident PSCI were significantly different by risk categories in internal (high, HR: 6.2, 95% CI: 4.1-9.3; Intermediate, HR: 2.7, 95% CI: 1.8-4.1) and external (high, HR: 6.1, 95% CI: 3.9-9.6; Intermediate, HR: 2.8, 95% CI: 1.9-4.3) validation cohorts.

CONCLUSION

Five-year risk of PSCI can be accurately predicted using routinely collected data. Model output can be used to risk stratify and identify individuals at increased risk for PSCI for preventive efforts.

DATA ACCESS STATEMENT

Mass General Brigham data contain protected health information and cannot be shared publicly. The data processing scripts used to perform analyses will be made available to interested researchers upon reasonable request to the corresponding author.

Predicting post-stroke cognitive impairment using electronic health record data

Importance

Secondary prevention interventions to reduce post-stroke cognitive impairment (PSCI) can be aided by the early identification of high-risk individuals who would benefit from risk factor modification.

Objective

To develop and evaluate a predictive model to identify patients at increased risk of PSCI over 5 years using data easily accessible from electronic health records.

Design

Cohort study with patients enrolled between 2003-2016 with follow-up through 2022.

Setting

Primary care practices affiliated with two academic medical centers.

Participants

Individuals 45 years or older, without prior stroke or prevalent cognitive impairment, with primary care visits and an incident ischemic stroke between 2003-2016 (development/internal validation cohort) or 2010-2022 (external validation cohort).

Exposures

Predictors of PSCI were ascertained from the electronic health record.

Main Outcome

The outcome was incident dementia/cognitive impairment within 5 years and beginning 3 months following stroke, ascertained using ICD-9/10 codes. For model variable selection, we considered potential predictors of PSCI and constructed 400 bootstrap samples with two-thirds of the model derivation sample. We ran 10-fold cross-validated Cox proportional hazards models using a least absolute shrinkage and selection operator (LASSO) penalty. Variables selected in >25% of samples were included.

Results

The analysis included 332 incident diagnoses of PSCI in the development cohort (n=3,741), and 161 and 128 incident diagnoses in the internal (n=1,925) and external (n=2,237) validation cohorts. The c-statistic for predicting PSCI was 0.731 (95% CI: 0.694-0.768) in the internal validation cohort, and 0.724 (95% CI: 0.681-0.766) in the external validation cohort. A risk score based on the beta coefficients of predictors from the development cohort stratified patients into low (0-7 points), intermediate (8-11 points), and high (12-35 points) risk groups. The hazard ratios for incident PSCI were significantly different by risk categories in internal (High, HR: 6.2, 95% CI 4.1-9.3; Intermediate, HR 2.7, 95% CI: 1.8-4.1) and external (High, HR: 6.1, 95% CI: 3.9-9.6; Intermediate, HR 2.8, 95% CI: 1.9-4.3) validation cohorts.

Conclusions and Relevance

Five-year risk of PSCI can be accurately predicted using routinely collected data. Model output can be used to risk stratify and identify individuals at increased risk for PSCI for preventive efforts.

Performance of Different Risk Scores for the Detection of Atrial Fibrillation Among Patients With Cryptogenic Stroke

BACKGROUND

Atrial fibrillation (AF) is a frequent underlying cause of cryptogenic stroke (CS) and its detection can be increased using implantable cardiac monitoring (ICM). We sought to evaluate different risk scores and assess their diagnostic ability in identifying patients with CS with underlying AF on ICM.

METHODS

Patients with CS, being admitted to a single tertiary stroke center between 2017 and 2022 and receiving ICM, were prospectively evaluated. The CHA2DS2-VASc, HAVOC, Brown ESUS-AF, and C2HEST scores were calculated at baseline. The primary outcome of interest was the detection of AF, which was defined as at least 1 AF episode on ICM lasting for 2 consecutive minutes or more. The diagnostic accuracy measures and the net reclassification improvement were calculated for the 4 risk scores. Stroke recurrence was evaluated as a secondary outcome.

RESULTS

A total of 250 patients with CS were included, and AF was detected by ICM in 20.4% (n=51) during a median monitoring period of 16 months. Patients with CS with AF detection were older compared with the rest (P=0.045). The median HAVOC, Brown ESUS-AF, and C2HEST scores were higher among the patients with AF compared with the patients without AF (all P<0.05), while the median CHA2DS2-VASc score was similar between the 2 groups. The corresponding C statistics for CHA2DS2-VASc, HAVOC, Brown ESUS-AF, and C2HEST for AF prediction were 0.576 (95% CI, 0.482-0.670), 0.612 (95% CI, 0.523-0.700), 0.666 (95% CI, 0.587-0.746), and 0.770 (95% CI, 0.699-0.839). The C2HEST score presented the highest diagnostic performance based on C statistics (P<0.05 after correction for multiple comparisons) and provided significant improvement in net reclassification for AF detection (>70%) compared with the other risk scores. Finally, stroke recurrence was documented in 5.6% of the study population, with no difference regarding the 4 risk scores between patients with and without recurrent stroke.

CONCLUSIONS

The C2HEST score was superior to the CHA2DS2-VASc, HAVOC, and Brown ESUS-AF scores for discriminating patients with CS with underlying AF using ICM.

Long-Term Cardiac Monitoring After Embolic Stroke of Undetermined Source: Search Longer, Look Harder

Embolic stroke of undetermined source (ESUS) represents a heterogeneous subgroup of patients with cryptogenic stroke, in which despite an extensive diagnostic workup the cause of stroke remains uncertain. Identifying covert atrial fibrillation among patients with ESUS remains challenging. The increasing use of cardiac implanted electronic devices (CIED), such as pacemakers, implantable defibrillators, and implantable loop recorders (ILR), has provided important information on the burden of subclinical atrial fibrillation. Accumulating evidence indicate that long-term continuous monitoring, especially in selected patients with ESUS, significantly increases the possibility of atrial fibrillation detection, suggesting it may be a cost-effective tool in secondary stroke prevention. This review summarizes available evidence related to the use of long-term cardiac monitoring and the use of implantable cardiac monitoring devices in patients with ESUS.

Natural Language Processing to Improve Prediction of Incident Atrial Fibrillation Using Electronic Health Records

Background Models predicting atrial fibrillation (AF) risk, such as Cohorts for Heart and Aging Research in Genomic Epidemiology AF (CHARGE-AF), have not performed as well in electronic health records. Natural language processing (NLP) may improve models by using narrative electronic health record text. Methods and Results From a primary care network, we included patients aged ≥65 years with visits between 2003 and 2013 in development (n=32 960) and internal validation cohorts (n=13 992). An external validation cohort from a separate network from 2015 to 2020 included 39 051 patients. Model features were defined using electronic health record codified data and narrative data with NLP. We developed 2 models to predict 5-year AF incidence using (1) codified+NLP data and (2) codified data only and evaluated model performance. The analysis included 2839 incident AF cases in the development cohort and 1057 and 2226 cases in internal and external validation cohorts, respectively. The C-statistic was greater (P<0.001) in codified+NLP model (0.744 [95% CI, 0.735-0.753]) compared with codified-only (0.730 [95% CI, 0.720-0.739]) in the development cohort. In internal validation, the C-statistic of codified+NLP was modestly higher (0.735 [95% CI, 0.720-0.749]) compared with codified-only (0.729 [95% CI, 0.715-0.744]; P=0.06) and CHARGE-AF (0.717 [95% CI, 0.703-0.731]; P=0.002). Codified+NLP and codified-only were well calibrated, whereas CHARGE-AF underestimated AF risk. In external validation, the C-statistic of codified+NLP (0.750 [95% CI, 0.740-0.760]) remained higher (P<0.001) than codified-only (0.738 [95% CI, 0.727-0.748]) and CHARGE-AF (0.735 [95% CI, 0.725-0.746]). Conclusions Estimation of 5-year risk of AF can be modestly improved using NLP to incorporate narrative electronic health record data.

Automated risk assessment of newly detected atrial fibrillation poststroke from electronic health record data using machine learning and natural language processing

Background

Timely detection of atrial fibrillation (AF) after stroke is highly clinically relevant, aiding decisions on the optimal strategies for secondary prevention of stroke. In the context of limited medical resources, it is crucial to set the right priorities of extended heart rhythm monitoring by stratifying patients into different risk groups likely to have newly detected AF (NDAF). This study aimed to develop an electronic health record (EHR)-based machine learning model to assess the risk of NDAF in an early stage after stroke.

Methods

Linked data between a hospital stroke registry and a deidentified research-based database including EHRs and administrative claims data was used. Demographic features, physiological measurements, routine laboratory results, and clinical free text were extracted from EHRs. The extreme gradient boosting algorithm was used to build the prediction model. The prediction performance was evaluated by the C-index and was compared to that of the AS5F and CHASE-LESS scores.

Results

The study population consisted of a training set of 4,064 and a temporal test set of 1,492 patients. During a median follow-up of 10.2 months, the incidence rate of NDAF was 87.0 per 1,000 person-year in the test set. On the test set, the model based on both structured and unstructured data achieved a C-index of 0.840, which was significantly higher than those of the AS5F (0.779, p = 0.023) and CHASE-LESS (0.768, p = 0.005) scores.

Conclusions

It is feasible to build a machine learning model to assess the risk of NDAF based on EHR data available at the time of hospital admission. Inclusion of information derived from clinical free text can significantly improve the model performance and may outperform risk scores developed using traditional statistical methods. Further studies are needed to assess the clinical usefulness of the prediction model.

Development and Validation of a Novel Score for Predicting Paroxysmal Atrial Fibrillation in Acute Ischemic Stroke

Atrial fibrillation (AF)—whether paroxysmal or sustained—increases the risk of stroke. We developed and validated a risk score for identifying patients at risk of paroxysmal atrial fibrillation (pAF) after acute ischemic stroke (AIS). A total of 6033 patients with AIS who received 24 h Holter monitoring were identified in the Chang Gung Research Database. Among the identified patients, 5290 with pAF and without AF were included in the multivariable logistic regression analysis to develop the pAF prediction model. The ABCD-SD score (Age, Systolic Blood pressure, Coronary artery disease, Dyslipidemia, and Standard Deviation of heart rate) comprises age (+2 points for every 10 years), systolic blood pressure (−1 point for every 20 mmHg), coronary artery disease (+2 points), dyslipidemia (−2 points), and standard deviation of heart rate (+2 points for every 3 beats per minute). Overall, 5.2% (274/5290) of patients had pAF. The pAF risk ranged from 0.8% (ABCD-SD score ≤ 7) to 18.3% (ABCD-SD score ≥ 15). The model achieved an area under the receiver operating characteristic curve (AUROCC) of 0.767 in the model development group. The ABCD-SD score could aid clinicians in identifying patients with AIS at risk of pAF for advanced cardiac monitoring.

Validation of Risk Scores for Predicting Atrial Fibrillation Detected After Stroke Based on an Electronic Medical Record Algorithm: A Registry-Claims-Electronic Medical Record Linked Data Study

Background

Poststroke atrial fibrillation (AF) screening aids decisions regarding the optimal secondary prevention strategies in patients with acute ischemic stroke (AIS). We used an electronic medical record (EMR) algorithm to identify AF in a cohort of AIS patients, which were used to validate eight risk scores for predicting AF detected after stroke (AFDAS).

Methods

We used linked data between a hospital stroke registry and a deidentified database including EMRs and administrative claims data. EMR algorithms were constructed to identify AF using diagnostic and medication codes as well as free clinical text. Based on the optimal EMR algorithm, the incidence rate of AFDAS was estimated. The predictive performance of 8 risk scores including AS5F, C₂HEST, CHADS₂, CHA₂DS₂-VASc, CHASE-LESS, HATCH, HAVOC, and Re-CHARGE-AF scores, were compared using the C-index, net reclassification improvement, integrated discrimination improvement, calibration curve, and decision curve analysis.

Results

The algorithm that defines AF as any positive mention of AF-related keywords in electrocardiography or echocardiography reports, or presence of diagnostic codes of AF was used to identify AF. Among the 5,412 AIS patients without known AF at stroke admission, the incidence rate of AFDAS was 84.5 per 1,000 person-year. The CHASE-LESS and AS5F scores were well calibrated and showed comparable C-indices (0.741 versus 0.730, p = 0.223), which were significantly higher than the other risk scores.

Conclusion

The CHASE-LESS and AS5F scores demonstrated adequate discrimination and calibration for predicting AFDAS. Both simple risk scores may help select patients for intensive AF monitoring.

Cohort design and natural language processing to reduce bias in electronic health records research

Electronic health record (EHR) datasets are statistically powerful but are subject to ascertainment bias and missingness. Using the Mass General Brigham multi-institutional EHR, we approximated a community-based cohort by sampling patients receiving longitudinal primary care between 2001-2018 (Community Care Cohort Project [C3PO], n = 520,868). We utilized natural language processing (NLP) to recover vital signs from unstructured notes. We assessed the validity of C3PO by deploying established risk models for myocardial infarction/stroke and atrial fibrillation. We then compared C3PO to Convenience Samples including all individuals from the same EHR with complete data, but without a longitudinal primary care requirement. NLP reduced the missingness of vital signs by 31%. NLP-recovered vital signs were highly correlated with values derived from structured fields (Pearson r range 0.95-0.99). Atrial fibrillation and myocardial infarction/stroke incidence were lower and risk models were better calibrated in C3PO as opposed to the Convenience Samples (calibration error range for myocardial infarction/stroke: 0.012-0.030 in C3PO vs. 0.028-0.046 in Convenience Samples; calibration error for atrial fibrillation 0.028 in C3PO vs. 0.036 in Convenience Samples). Sampling patients receiving regular primary care and using NLP to recover missing data may reduce bias and maximize generalizability of EHR research.