Patent Foramen Ovale Closure in Older Patients With Stroke: Patient Selection for Trial Feasibility

BACKGROUND AND OBJECTIVES

Whether patent foramen ovale (PFO) closure benefits older patients with PFO and cryptogenic stroke is unknown because randomized controlled trials (RCTs) have predominantly enrolled patients younger than 60 years of age. Our objective was to estimate anticipated effects of PFO closure in older patients to predict the numbers needed to plan an RCT.

METHODS

Effectiveness estimates are derived from major observational studies (Risk of Paradoxical Embolism [RoPE] Study and Oxford Vascular Study, together referred to as the "RoPE-Ox" database) and all 6 major RCTs (Systematic, Collaborative, PFO Closure Evaluation [SCOPE] Consortium). To estimate stroke recurrence risk, observed outcomes were calculated for patients older than 60 years in the age-inclusive observational databases (n = 549). To estimate the reduction in the rate of recurrent stroke associated with PFO closure vs medical therapy based on the RoPE score and the presence of high-risk PFO features, a Cox proportional hazards regression model was developed on the RCT data in the SCOPE database (n = 3,740). These estimates were used to calculate sample sizes required for a future RCT.

RESULTS

Five-year risk of stroke recurrence using Kaplan-Meier estimates was 13.7 (95% CI 10.5-17.9) overall, 14.9% (95% CI 10.2-21.6) in those with high-risk PFO features. Predicted relative reduction in the event rate with PFO closure was 12.9% overall, 48.8% in those with a high-risk PFO feature. Using these estimates, enrolling all older patients with cryptogenic stroke and PFO would require much larger samples than those used for prior PFO closure trials, but selectively enrolling patients with high-risk PFO features would require totals of 630 patients for 90% power and 471 patients for 80% power, with an average of 5 years of follow-up.

DISCUSSION

Based on our projections, anticipated effect sizes in older patients with high-risk features make a trial in these subjects feasible. With lengthening life expectancy in almost all regions of the world, the utility of PFO closure in older adults is increasingly important to explore.

Is Incidentally Discovered Covert Cerebrovascular Disease Ignorable?

This Viewpoint discusses the clinical implications of incidentally discovered covert cerebrovascular disease.

Mortality Outcomes in a Large Population with and without Covert Cerebrovascular Disease

Covert cerebrovascular disease (CCD) is frequently reported on neuroimaging and associates with increased dementia and stroke risk. We aimed to determine how incidentally-discovered CCD during clinical neuroimaging in a large population associates with mortality. We screened CT and MRI reports of adults aged ≥50 in the Kaiser Permanente Southern California health system who underwent neuroimaging for a non-stroke clinical indication from 2009-2019. Natural language processing identified incidental covert brain infarcts (CBI) and/or white matter hyperintensities (WMH), grading WMH as mild/moderate/severe. Models adjusted for age, sex, ethnicity, multimorbidity, vascular risks, depression, exercise, and imaging modality. Of n=241,028, the mean age was 64.9 (SD=10.4); mean follow-up 4.46 years; 178,554 (74.1%) had CT; 62,474 (25.9%) had MRI; 11,328 (4.7%) had CBI; and 69,927 (29.0%) had WMH. The mortality rate per 1,000 person-years with CBI was 59.0 (95%CI 57.0-61.1); with WMH=46.5 (45.7-47.2); with neither=17.4 (17.1-17.7). In adjusted models, mortality risk associated with CBI was modified by age, e.g. HR 1.34 [1.21-1.48] at age 56.1 years vs HR 1.22 [1.17-1.28] at age 72 years. Mortality associated with WMH was modified by both age and imaging modality e.g., WMH on MRI at age 56.1 HR = 1.26 [1.18-1.35]; WMH on MRI at age 72 HR 1.15 [1.09-1.21]; WMH on CT at age 56.1 HR 1.41 [1.33-1.50]; WMH on CT at age 72 HR 1.28 [1.24-1.32], vs. patients without CBI or without WMH, respectively. Increasing WMH severity associated with higher mortality, e.g. mild WMH on MRI had adjusted HR=1.13 [1.06-1.20] while severe WMH on CT had HR=1.45 [1.33-1.59]. Incidentally-detected CBI and WMH on population-based clinical neuroimaging can predict higher mortality rates. We need treatments and healthcare planning for individuals with CCD.

A Clinical Model to Predict the Occurrence of Select High-risk Infections in the First Year Following Heart Transplantation

Background

Invasive infection remains a dangerous complication of heart transplantation (HT). No objectively defined set of clinical risk factors has been established to reliably predict infection in HT. The aim of this study was to develop a clinical prediction model for use at 1 mo post-HT to predict serious infection by 1 y.

Methods

A retrospective cohort study of HT recipients (2000-2018) was performed. The composite endpoint included cytomegalovirus (CMV), herpes simplex or varicella zoster virus infection, blood stream infection, invasive fungal, or nocardial infection occurring 1 mo to 1 y post-HT. A least absolute shrinkage and selection operator regression model was constructed using 10 candidate variables. A concordance statistic, calibration curve, and mean calibration error were calculated. A scoring system was derived for ease of clinical application.

Results

Three hundred seventy-five patients were analyzed; 93 patients experienced an outcome event. All variables remained in the final model: aged 55 y or above, history of diabetes, need for renal replacement therapy in first month, CMV risk derived from donor and recipient serology, use of induction and/or early lymphodepleting therapy in the first month, use of trimethoprim-sulfamethoxazole prophylaxis at 1 mo, lymphocyte count under 0.75 × 103cells/µL at 1 mo, and inpatient status at 1 mo. Good discrimination (C-index 0.80) and calibration (mean absolute calibration error 3.6%) were demonstrated.

Conclusion

This model synthesizes multiple highly relevant clinical parameters, available at 1 mo post-HT, into a unified, objective, and clinically useful prediction tool for occurrence of serious infection by 1 y post-HT.

Management of covert brain infarction survey: A call to care for and trial this neglected population

BACKGROUND

Covert brain infarction (CBI) is highly prevalent and linked with stroke risk factors, increased mortality, and morbidity. Evidence to guide management is sparse. We sought to gain information on current practice and attitudes toward CBI and to compare differences in management according to CBI phenotype.

METHODS

We conducted a web-based, structured, international survey from November 2021 to February 2022 among neurologists and neuroradiologists. The survey captured respondents' baseline characteristics, general approach toward CBI and included two case scenarios designed to evaluate management decisions taken upon incidental detection of an embolic-phenotype and a small-vessel-disease phenotype.

RESULTS

Of 627 respondents (38% vascular neurologists, 24% general neurologists, and 26% neuroradiologists), 362 (58%) had a partial, and 305 (49%) a complete response. Most respondents were university hospital senior faculty members experienced in stroke, mostly from Europe and Asia. Only 66 (18%) of respondents had established institutional written protocols to manage CBI. The majority indicated that they were uncertain regarding useful investigations and further management of CBI patients (median 67 on a slider 0-100, 95% CI 35-81). Almost all respondents (97%) indicated that they would assess vascular risk factors. Although most would investigate and treat similarly to ischemic stroke for both phenotypes, including initiating antithrombotic treatment, there was considerable diagnostic and therapeutic heterogeneity. Less than half of respondents (42%) would assess cognitive function or depression.

CONCLUSIONS

There is a high degree of uncertainty and heterogeneity regarding management of two common types of CBI, even among experienced stroke physicians. Respondents were more proactive regarding the diagnostic and therapeutic management than the minimum recommended by current expert opinions. More data are required to guide management of CBI; meantime, more consistent approaches to identification and consistent application of current knowledge, that also consider cognition and mood, would be promising first steps to improve consistency of care.

Stratifying future stroke risk with incidentally-discovered white matter disease severity and covert brain infarct site

BACKGROUND

Covert cerebrovascular disease (CCD) includes white matter disease (WMD) and covert brain infarction (CBI). Incidentally-discovered CCD is associated with increased risk of subsequent symptomatic stroke. However, it is unknown whether the severity of WMD or the location of CBI predicts risk.

OBJECTIVES

To examine the association of incidentally-discovered WMD severity and CBI location with risk of subsequent symptomatic stroke.

METHOD

This retrospective cohort study includes patients 50 years old in the Kaiser Permanente Southern California health system who received neuroimaging for a non-stroke indication between 2009-2019. Incidental CBI and WMD were identified via natural language processing of the neuroimage report, and WMD severity was classified into grades.

RESULTS

261,960 patients received neuroimaging; 78,555 (30.0%) were identified to have incidental WMD, and 12,857 (4.9%) to have incidental CBI. Increasing WMD severity is associated with increased incidence rate of future stroke. However, the stroke incidence rate in CT-identified WMD is higher at each level of severity compared to rates in MRI-identified WMD. Patients with mild WMD via CT have a stroke incidence rate of 24.9 per 1,000 person-years, similar to that of patients with severe WMD via MRI. Among incidentally-discovered CBI patients with a determined CBI location, 97.9% are subcortical rather than cortical infarcts. CBI confers a similar risk of future stroke, whether cortical or subcortical, or whether MRI- or CT-detected.

CONCLUSIONS

Increasing severity of incidental WMD is associated with an increased risk of future symptomatic stroke, dependent on the imaging modality. Subcortical and cortical CBI conferred similar risks.

Integrated care in patients with atrial fibrillation- a predictive heterogeneous treatment effect analysis of the ALL-IN trial

INTRODUCTION

Integrated care is effective in reducing all-cause mortality in patients with atrial fibrillation (AF) in primary care, though time and resource intensive. The aim of the current study was to assess whether integrated care should be directed at all AF patients equally.

METHODS

The ALL-IN trial (n = 1,240 patients, median age 77 years) was a cluster-randomized trial in which primary care practices were randomized to provide integrated care or usual care to AF patients aged 65 years and older. Integrated care comprised of (i) anticoagulation monitoring, (ii) quarterly checkups and (iii) easy-access consultation with cardiologists. For the current analysis, cox proportional hazard analysis with all clinical variables from the CHA2DS2-VASc score was used to predict all-cause mortality in the ALL-IN trial. Subsequently, the hazard ratio and absolute risk reduction were plotted as a function of this predicted mortality risk to explore treatment heterogeneity.

RESULTS

Under usual care, after a median of 2 years follow-up the absolute risk of all-cause mortality in the highest-risk quarter was 31.0%, compared to 4.6% in the lowest-risk quarter. On the relative scale, there was no evidence of treatment heterogeneity (p for interaction = 0.90). However, there was substantial treatment heterogeneity on the absolute scale: risk reduction in the lowest risk- quarter of risk 3.3% (95% CI -0.4% - 7.0) compared to 12.0% (95% CI 2.7% - 22.0) in the highest risk quarter.

CONCLUSION

While the relative degree of benefit from integrated AF care is similar in all patients, patients with a high all-cause mortality risk have a greater benefit on an absolute scale and should therefore be prioritized when implementing integrated care.

Assessing heterogeneity of treatment effect in multiple sclerosis trials

Multiple sclerosis (MS) is heterogeneous with respect to outcomes, and evaluating possible heterogeneity of treatment effect (HTE) is of high interest. HTE is non-random variation in the magnitude of a treatment effect on a clinical outcome across levels of a covariate (i.e. a patient attribute or set of attributes). Multiple statistical techniques can evaluate HTE. The simplest but most bias-prone is conventional one variable-at-a-time subgroup analysis. Recently, multivariable predictive approaches have been promoted to provide more patient-centered results, by accounting for multiple relevant attributes simultaneously. We review approaches used to estimate HTE in clinical trials of MS.

Performance metrics for models designed to predict treatment effect

BACKGROUND

Measuring the performance of models that predict individualized treatment effect is challenging because the outcomes of two alternative treatments are inherently unobservable in one patient. The C-for-benefit was proposed to measure discriminative ability. However, measures of calibration and overall performance are still lacking. We aimed to propose metrics of calibration and overall performance for models predicting treatment effect in randomized clinical trials (RCTs).

METHODS

Similar to the previously proposed C-for-benefit, we defined observed pairwise treatment effect as the difference between outcomes in pairs of matched patients with different treatment assignment. We match each untreated patient with the nearest treated patient based on the Mahalanobis distance between patient characteristics. Then, we define the Eavg-for-benefit, E50-for-benefit, and E90-for-benefit as the average, median, and 90th quantile of the absolute distance between the predicted pairwise treatment effects and local-regression-smoothed observed pairwise treatment effects. Furthermore, we define the cross-entropy-for-benefit and Brier-for-benefit as the logarithmic and average squared distance between predicted and observed pairwise treatment effects. In a simulation study, the metric values of deliberately "perturbed models" were compared to those of the data-generating model, i.e., "optimal model". To illustrate these performance metrics, different modeling approaches for predicting treatment effect are applied to the data of the Diabetes Prevention Program: 1) a risk modelling approach with restricted cubic splines; 2) an effect modelling approach including penalized treatment interactions; and 3) the causal forest.

RESULTS

As desired, performance metric values of "perturbed models" were consistently worse than those of the "optimal model" (Eavg-for-benefit ≥ 0.043 versus 0.002, E50-for-benefit ≥ 0.032 versus 0.001, E90-for-benefit ≥ 0.084 versus 0.004, cross-entropy-for-benefit ≥ 0.765 versus 0.750, Brier-for-benefit ≥ 0.220 versus 0.218). Calibration, discriminative ability, and overall performance of three different models were similar in the case study. The proposed metrics were implemented in a publicly available R-package "HTEPredictionMetrics".

CONCLUSION

The proposed metrics are useful to assess the calibration and overall performance of models predicting treatment effect in RCTs.

Systematic metareview of prediction studies demonstrates stable trends in bias and low PROBAST inter-rater agreement

OBJECTIVES

To (1) explore trends of risk of bias (ROB) in prediction research over time following key methodological publications, using the Prediction model Risk Of Bias ASsessment Tool (PROBAST) and (2) assess the inter-rater agreement of the PROBAST.

STUDY DESIGN AND SETTING

PubMed and Web of Science were searched for reviews with extractable PROBAST scores on domain and signaling question (SQ) level. ROB trends were visually correlated with yearly citations of key publications. Inter-rater agreement was assessed using Cohen's Kappa.

RESULTS

One hundred and thirty nine systematic reviews were included, of which 85 reviews (containing 2,477 single studies) on domain level and 54 reviews (containing 2,458 single studies) on SQ level. High ROB was prevalent, especially in the Analysis domain, and overall trends of ROB remained relatively stable over time. The inter-rater agreement was low, both on domain (Kappa 0.04-0.26) and SQ level (Kappa -0.14 to 0.49).

CONCLUSION

Prediction model studies are at high ROB and time trends in ROB as assessed with the PROBAST remain relatively stable. These results might be explained by key publications having no influence on ROB or recency of key publications. Moreover, the trend may suffer from the low inter-rater agreement and ceiling effect of the PROBAST. The inter-rater agreement could potentially be improved by altering the PROBAST or providing training on how to apply the PROBAST.

Treatment effect modification due to comorbidity: Individual participant data meta-analyses of 120 randomised controlled trials

BACKGROUND

People with comorbidities are underrepresented in clinical trials. Empirical estimates of treatment effect modification by comorbidity are lacking, leading to uncertainty in treatment recommendations. We aimed to produce estimates of treatment effect modification by comorbidity using individual participant data (IPD).

METHODS AND FINDINGS

We obtained IPD for 120 industry-sponsored phase 3/4 trials across 22 index conditions (n = 128,331). Trials had to be registered between 1990 and 2017 and have recruited ≥300 people. Included trials were multicentre and international. For each index condition, we analysed the outcome most frequently reported in the included trials. We performed a two-stage IPD meta-analysis to estimate modification of treatment effect by comorbidity. First, for each trial, we modelled the interaction between comorbidity and treatment arm adjusted for age and sex. Second, for each treatment within each index condition, we meta-analysed the comorbidity-treatment interaction terms from each trial. We estimated the effect of comorbidity measured in 3 ways: (i) the number of comorbidities (in addition to the index condition); (ii) presence or absence of the 6 commonest comorbid diseases for each index condition; and (iii) using continuous markers of underlying conditions (e.g., estimated glomerular filtration rate (eGFR)). Treatment effects were modelled on the usual scale for the type of outcome (absolute scale for numerical outcomes, relative scale for binary outcomes). Mean age in the trials ranged from 37.1 (allergic rhinitis trials) to 73.0 (dementia trials) and percentage of male participants range from 4.4% (osteoporosis trials) to 100% (benign prostatic hypertrophy trials). The percentage of participants with 3 or more comorbidities ranged from 2.3% (allergic rhinitis trials) to 57% (systemic lupus erythematosus trials). We found no evidence of modification of treatment efficacy by comorbidity, for any of the 3 measures of comorbidity. This was the case for 20 conditions for which the outcome variable was continuous (e.g., change in glycosylated haemoglobin in diabetes) and for 3 conditions in which the outcomes were discrete events (e.g., number of headaches in migraine). Although all were null, estimates of treatment effect modification were more precise in some cases (e.g., sodium-glucose co-transporter-2 (SGLT2) inhibitors for type 2 diabetes-interaction term for comorbidity count 0.004, 95% CI -0.01 to 0.02) while for others credible intervals were wide (e.g., corticosteroids for asthma-interaction term -0.22, 95% CI -1.07 to 0.54). The main limitation is that these trials were not designed or powered to assess variation in treatment effect by comorbidity, and relatively few trial participants had >3 comorbidities.

CONCLUSIONS

Assessments of treatment effect modification rarely consider comorbidity. Our findings demonstrate that for trials included in this analysis, there was no empirical evidence of treatment effect modification by comorbidity. The standard assumption used in evidence syntheses is that efficacy is constant across subgroups, although this is often criticised. Our findings suggest that for modest levels of comorbidities, this assumption is reasonable. Thus, trial efficacy findings can be combined with data on natural history and competing risks to assess the likely overall benefit of treatments in the context of comorbidity.

Overall average treatment effects from clinical trials, one-variable-at-a-time subgroup analyses and predictive approaches to heterogeneous treatment effects: Toward a more patient-centered evidence-based medicine

Despite the predominance of the evidence-based medicine paradigm, a fundamental incongruity remains: Evidence is derived from groups of people, yet medical decisions are made by and for individuals. Randomization ensures the comparability of treatment groups within a clinical trial, which allows for unbiased estimation of average treatment effects. If we treated groups of patients instead of individuals, or if patients with the same disease were identical to one another in all factors that determined the harms and the benefits of therapy, then these group-level averages would make a perfectly sound foundation for medical decision-making. But patients differ from one another in many ways that determine the likelihood of an outcome, both with and without a treatment. Nevertheless, popular approaches to evidence-based medicine have encouraged a reliance on the average treatment effects estimated from clinical trials and meta-analysis as guides to decision-making for individuals. Here, we discuss the limitations of this approach as well as limitations of conventional, one-variable-at-a-time subgroup analysis; finally, we discuss the rationale for "predictive" approaches to heterogeneous treatment effects. Predictive approaches to heterogeneous treatment effects combine methods for causal inference (e.g. randomization) with methods for prediction that permit inferences about which patients are likely to benefit and which are not, taking into account multiple relevant variables simultaneously to yield "personalized" estimates of benefit-harm trade-offs. We focus on risk modeling approaches, which rely on the mathematical dependence of the absolute treatment effect with the baseline risk, which varies substantially "across patients" in most trials. While there are a number of examples of risk modeling approaches that have been practice-changing, risk modeling does not provide ideal estimates of individual treatment effects, since risk modeling does not account for how individual variables might modify the effects of therapy. In "effect modeling," prediction models are developed directly on clinical trial data, including terms for treatment and treatment effect interactions. These more flexible approaches may better uncover individualized treatment effects, but are also prone to overfitting when dimensionality is high, power is low, and there is limited prior knowledge about effect modifiers.

Association of Preexisting Heart Failure With Outcomes in Older Patients With Diffuse Large B-Cell Lymphoma

Importance

Anthracycline-containing regimens are highly effective for diffuse large B-cell lymphoma (DLBCL); however, patients with preexisting heart failure (HF) may be less likely to receive anthracyclines and may be at higher risk of lymphoma mortality.

Objective

To assess the prevalence of preexisting HF in older patients with DLBCL and its association with treatment patterns and outcomes.

Design, Setting, and Participants

This longitudinal cohort study used data from the Surveillance, Epidemiology, and End Results (SEER)-Medicare registry from 1999 to 2016. The SEER registry is a system of population-based cancer registries, capturing more than 25% of the US population. Linkage to Medicare offers additional information from billing claims. This study included individuals 65 years and older with newly diagnosed DLBCL from 2000 to 2015 with Medicare Part A or B continuously in the year prior to lymphoma diagnosis. Data were analyzed from September 2020 to December 2022.

Exposures

Preexisting HF in the year prior to DLBCL diagnosis ascertained from billing codes required one of the following: (1) 1 primary inpatient discharge diagnosis, (2) 2 outpatient diagnoses, (3) 3 secondary inpatient discharge diagnoses, (4) 3 emergency department diagnoses, or (5) 2 secondary inpatient discharge diagnoses plus 1 outpatient diagnosis.

Main Outcomes and Measures

The primary outcome was anthracycline-based treatment. The secondary outcomes were (1) cardioprotective medications and (2) cause-specific mortality. The associations between preexisting HF and cancer treatment were estimated using multivariable logistic regression. The associations between preexisting HF and cause-specific mortality were evaluated using cause-specific Cox proportional hazards models with adjustment for comorbidities and cancer treatment.

Results

Of 30 728 included patients with DLBCL, 15 474 (50.4%) were female, and the mean (SD) age was 77.8 (7.2) years. Preexisting HF at lymphoma diagnosis was present in 4266 patients (13.9%). Patients with preexisting HF were less likely to be treated with an anthracycline (odds ratio, 0.55; 95% CI, 0.49-0.61). Among patients with preexisting HF who received an anthracycline, dexrazoxane or liposomal doxorubicin were used in 78 of 1119 patients (7.0%). One-year lymphoma mortality was 41.8% (95% CI, 40.5-43.2) with preexisting HF and 29.6% (95% CI, 29.0%-30.1%) without preexisting HF. Preexisting HF was associated with higher lymphoma mortality in models adjusting for baseline and time-varying treatment factors (hazard ratio, 1.24; 95% CI, 1.18-1.31).

Conclusions and Relevance

In this study, preexisting HF in patients with newly diagnosed DLBCL was common and was associated with lower use of anthracyclines and lower use of any chemotherapy. Trials are needed for this high-risk population.

A standardized framework for risk-based assessment of treatment effect heterogeneity in observational healthcare databases

Treatment effects are often anticipated to vary across groups of patients with different baseline risk. The Predictive Approaches to Treatment Effect Heterogeneity (PATH) statement focused on baseline risk as a robust predictor of treatment effect and provided guidance on risk-based assessment of treatment effect heterogeneity in a randomized controlled trial. The aim of this study is to extend this approach to the observational setting using a standardized scalable framework. The proposed framework consists of five steps: (1) definition of the research aim, i.e., the population, the treatment, the comparator and the outcome(s) of interest; (2) identification of relevant databases; (3) development of a prediction model for the outcome(s) of interest; (4) estimation of relative and absolute treatment effect within strata of predicted risk, after adjusting for observed confounding; (5) presentation of the results. We demonstrate our framework by evaluating heterogeneity of the effect of thiazide or thiazide-like diuretics versus angiotensin-converting enzyme inhibitors on three efficacy and nine safety outcomes across three observational databases. We provide a publicly available R software package for applying this framework to any database mapped to the Observational Medical Outcomes Partnership Common Data Model. In our demonstration, patients at low risk of acute myocardial infarction receive negligible absolute benefits for all three efficacy outcomes, though they are more pronounced in the highest risk group, especially for acute myocardial infarction. Our framework allows for the evaluation of differential treatment effects across risk strata, which offers the opportunity to consider the benefit-harm trade-off between alternative treatments.

Estimating individualized treatment effects from randomized controlled trials: a simulation study to compare risk-based approaches

BACKGROUND

Baseline outcome risk can be an important determinant of absolute treatment benefit and has been used in guidelines for "personalizing" medical decisions. We compared easily applicable risk-based methods for optimal prediction of individualized treatment effects.

METHODS

We simulated RCT data using diverse assumptions for the average treatment effect, a baseline prognostic index of risk, the shape of its interaction with treatment (none, linear, quadratic or non-monotonic), and the magnitude of treatment-related harms (none or constant independent of the prognostic index). We predicted absolute benefit using: models with a constant relative treatment effect; stratification in quarters of the prognostic index; models including a linear interaction of treatment with the prognostic index; models including an interaction of treatment with a restricted cubic spline transformation of the prognostic index; an adaptive approach using Akaike's Information Criterion. We evaluated predictive performance using root mean squared error and measures of discrimination and calibration for benefit.

RESULTS

The linear-interaction model displayed optimal or close-to-optimal performance across many simulation scenarios with moderate sample size (N = 4,250; ~ 785 events). The restricted cubic splines model was optimal for strong non-linear deviations from a constant treatment effect, particularly when sample size was larger (N = 17,000). The adaptive approach also required larger sample sizes. These findings were illustrated in the GUSTO-I trial.

CONCLUSIONS

An interaction between baseline risk and treatment assignment should be considered to improve treatment effect predictions.

US and Dutch Perspectives on the Use of COVID-19 Clinical Prediction Models: Findings from a Qualitative Analysis

INTRODUCTION

Clinical prediction models (CPMs) for coronavirus disease 2019 (COVID-19) may support clinical decision making, treatment, and communication. However, attitudes about using CPMs for COVID-19 decision making are unknown.

METHODS

Online focus groups and interviews were conducted among health care providers, survivors of COVID-19, and surrogates (i.e., loved ones/surrogate decision makers) in the United States and the Netherlands. Semistructured questions explored experiences about clinical decision making in COVID-19 care and facilitators and barriers for implementing CPMs.

RESULTS

In the United States, we conducted 4 online focus groups with 1) providers and 2) surrogates and survivors of COVID-19 between January 2021 and July 2021. In the Netherlands, we conducted 3 focus groups and 4 individual interviews with 1) providers and 2) surrogates and survivors of COVID-19 between May 2021 and July 2021. Providers expressed concern about CPM validity and the belief that patients may interpret CPM predictions as absolute. They described CPMs as potentially useful for resource allocation, triaging, education, and research. Several surrogates and people who had COVID-19 were not given prognostic estimates but believed this information would have supported and influenced their decision making. A limited number of participants felt the data would not have applied to them and that they or their loved ones may not have survived, as poor prognosis may have suggested withdrawal of treatment.

CONCLUSIONS

Many providers had reservations about using CPMs for people with COVID-19 due to concerns about CPM validity and patient-level interpretation of the outcome predictions. However, several people who survived COVID-19 and their surrogates indicated that they would have found this information useful for decision making. Therefore, information provision may be needed to improve provider-level comfort and patient and surrogate understanding of CPMs.

HIGHLIGHTS

While clinical prediction models (CPMs) may provide an objective means of assessing COVID-19 prognosis, provider concerns about CPM validity and the interpretation of CPM predictions may limit their clinical use.Providers felt that CPMs may be most useful for resource allocation, triage, research, or educational purposes for COVID-19.Several survivors of COVID-19 and their surrogates felt that CPMs would have been informative and may have aided them in making COVID-19 treatment decisions, while others felt the data would not have applied to them.

Comorbidity and health-related quality of life in people with a chronic medical condition in randomised clinical trials: An individual participant data meta-analysis

BACKGROUND

Health-related quality of life metrics evaluate treatments in ways that matter to patients, so are often included in randomised clinical trials (hereafter trials). Multimorbidity, where individuals have 2 or more conditions, is negatively associated with quality of life. However, whether multimorbidity predicts change over time or modifies treatment effects for quality of life is unknown. Therefore, clinicians and guideline developers are uncertain about the applicability of trial findings to people with multimorbidity. We examined whether comorbidity count (higher counts indicating greater multimorbidity) (i) is associated with quality of life at baseline; (ii) predicts change in quality of life over time; and/or (iii) modifies treatment effects on quality of life.

METHODS AND FINDINGS

Included trials were registered on the United States trials registry for selected index medical conditions and drug classes, phase 2/3, 3 or 4, had ≥300 participants, a nonrestrictive upper age limit, and were available on 1 of 2 trial repositories on 21 November 2016 and 18 May 2018, respectively. Of 124 meeting these criteria, 56 trials (33,421 participants, 16 index conditions, and 23 drug classes) collected a generic quality of life outcome measure (35 EuroQol-5 dimension (EQ-5D), 31 36-item short form survey (SF-36) with 10 collecting both). Blinding and completeness of follow up were examined for each trial. Using trials where individual participant data (IPD) was available from 2 repositories, a comorbidity count was calculated from medical history and/or prescriptions data. Linear regressions were fitted for the association between comorbidity count and (i) quality of life at baseline; (ii) change in quality of life during trial follow up; and (iii) treatment effects on quality of life. These results were then combined in Bayesian linear models. Posterior samples were summarised via the mean, 2.5th and 97.5th percentiles as credible intervals (95% CI) and via the proportion with values less than 0 as the probability (PBayes) of a negative association. All results are in standardised units (obtained by dividing the EQ-5D/SF-36 estimates by published population standard deviations). Per additional comorbidity, adjusting for age and sex, across all index conditions and treatment comparisons, comorbidity count was associated with lower quality of life at baseline and with a decline in quality of life over time (EQ-5D -0.02 [95% CI -0.03 to -0.01], PBayes > 0.999). Associations were similar, but with wider 95% CIs crossing the null for SF-36-PCS and SF-36-MCS (-0.05 [-0.10 to 0.01], PBayes = 0.956 and -0.05 [-0.10 to 0.01], PBayes = 0.966, respectively). Importantly, there was no evidence of any interaction between comorbidity count and treatment efficacy for either EQ-5D or SF-36 (EQ-5D -0.0035 [95% CI -0.0153 to -0.0065], PBayes = 0.746; SF-36-MCS (-0.0111 [95% CI -0.0647 to 0.0416], PBayes = 0.70 and SF-36-PCS -0.0092 [95% CI -0.0758 to 0.0476], PBayes = 0.631.

CONCLUSIONS

Treatment effects on quality of life did not differ by multimorbidity (measured via a comorbidity count) at baseline-for the medical conditions studied, types and severity of comorbidities and level of quality of life at baseline, suggesting that evidence from clinical trials is likely to be applicable to settings with (at least modestly) higher levels of comorbidity.

TRIAL REGISTRATION

A prespecified protocol was registered on PROSPERO (CRD42018048202).

Association of Incidentally Discovered Covert Cerebrovascular Disease Identified Using Natural Language Processing and Future Dementia

Background Covert cerebrovascular disease (CCD) has been shown to be associated with dementia in population-based studies with magnetic resonance imaging (MRI) screening, but dementia risk associated with incidentally discovered CCD is not known. Methods and Results Individuals aged ≥50 years enrolled in the Kaiser Permanente Southern California health system receiving head computed tomography (CT) or MRI for nonstroke indications from 2009 to 2019, without prior ischemic stroke/transient ischemic attack, dementia/Alzheimer disease, or visit reason/scan indication suggestive of cognitive decline or stroke were included. Natural language processing identified incidentally discovered covert brain infarction (id-CBI) and white matter disease (id-WMD) on the neuroimage report; white matter disease was characterized as mild, moderate, severe, or undetermined. We estimated risk of dementia associated with id-CBI and id-WMD. Among 241 050 qualified individuals, natural language processing identified 69 931 (29.0%) with id-WMD and 11 328 (4.7%) with id-CBI. Dementia incidence rates (per 1000 person-years) were 23.5 (95% CI, 22.9-24.0) for patients with id-WMD, 29.4 (95% CI, 27.9-31.0) with id-CBI, and 6.0 (95% CI, 5.8-6.2) without id-CCD. The association of id-WMD with future dementia was stronger in younger (aged <70 years) versus older (aged ≥70 years) patients and for CT- versus MRI-discovered lesions. For patients with versus without id-WMD on CT, the adjusted HR was 2.87 (95% CI, 2.58-3.19) for older and 1.87 (95% CI, 1.79-1.95) for younger patients. For patients with versus without id-WMD on MRI, the adjusted HR for dementia risk was 2.28 (95% CI, 1.99-2.62) for older and 1.48 (95% CI, 1.32-1.66) for younger patients. The adjusted HR for id-CBI was 2.02 (95% CI, 1.70-2.41) for older and 1.22 (95% CI, 1.15-1.30) for younger patients for either modality. Dementia risk was strongly correlated with id-WMD severity; adjusted HRs compared with patients who were negative for id-WMD by MRI ranged from 1.41 (95% CI, 1.25-1.60) for those with mild disease on MRI to 4.11 (95% CI, 3.58-4.72) for those with severe disease on CT. Conclusions Incidentally discovered CCD is common and associated with a high risk of dementia, representing an opportunity for prevention. The association is strengthened when discovered at younger age, by increasing id-WMD severity, and when id-WMD is detected by CT scan rather than MRI.

Prognostic models for COVID-19 needed updating to warrant transportability over time and space

BACKGROUND

Supporting decisions for patients who present to the emergency department (ED) with COVID-19 requires accurate prognostication. We aimed to evaluate prognostic models for predicting outcomes in hospitalized patients with COVID-19, in different locations and across time.

METHODS

We included patients who presented to the ED with suspected COVID-19 and were admitted to 12 hospitals in the New York City (NYC) area and 4 large Dutch hospitals. We used second-wave patients who presented between September and December 2020 (2137 and 3252 in NYC and the Netherlands, respectively) to evaluate models that were developed on first-wave patients who presented between March and August 2020 (12,163 and 5831). We evaluated two prognostic models for in-hospital death: The Northwell COVID-19 Survival (NOCOS) model was developed on NYC data and the COVID Outcome Prediction in the Emergency Department (COPE) model was developed on Dutch data. These models were validated on subsequent second-wave data at the same site (temporal validation) and at the other site (geographic validation). We assessed model performance by the Area Under the receiver operating characteristic Curve (AUC), by the E-statistic, and by net benefit.

RESULTS

Twenty-eight-day mortality was considerably higher in the NYC first-wave data (21.0%), compared to the second-wave (10.1%) and the Dutch data (first wave 10.8%; second wave 10.0%). COPE discriminated well at temporal validation (AUC 0.82), with excellent calibration (E-statistic 0.8%). At geographic validation, discrimination was satisfactory (AUC 0.78), but with moderate over-prediction of mortality risk, particularly in higher-risk patients (E-statistic 2.9%). While discrimination was adequate when NOCOS was tested on second-wave NYC data (AUC 0.77), NOCOS systematically overestimated the mortality risk (E-statistic 5.1%). Discrimination in the Dutch data was good (AUC 0.81), but with over-prediction of risk, particularly in lower-risk patients (E-statistic 4.0%). Recalibration of COPE and NOCOS led to limited net benefit improvement in Dutch data, but to substantial net benefit improvement in NYC data.

CONCLUSIONS

NOCOS performed moderately worse than COPE, probably reflecting unique aspects of the early pandemic in NYC. Frequent updating of prognostic models is likely to be required for transportability over time and space during a dynamic pandemic.

Development and validation of self-monitoring auto-updating prognostic models of survival for hospitalized COVID-19 patients

Clinical prognostic models can assist patient care decisions. However, their performance can drift over time and location, necessitating model monitoring and updating. Despite rapid and significant changes during the pandemic, prognostic models for COVID-19 patients do not currently account for these drifts. We develop a framework for continuously monitoring and updating prognostic models and apply it to predict 28-day survival in COVID-19 patients. We use demographic, laboratory, and clinical data from electronic health records of 34912 hospitalized COVID-19 patients from March 2020 until May 2022 and compare three modeling methods. Model calibration performance drift is immediately detected with minor fluctuations in discrimination. The overall calibration on the prospective validation cohort is significantly improved when comparing the dynamically updated models against their static counterparts. Our findings suggest that, using this framework, models remain accurate and well-calibrated across various waves, variants, race and sex and yield positive net-benefits.

Association of Atrial Septal Aneurysm and Shunt Size With Stroke Recurrence and Benefit From Patent Foramen Ovale Closure

Importance

The Patent Foramen Ovale (PFO)-Associated Stroke Causal Likelihood classification system combines information regarding noncardiac patient features (vascular risk factors, infarct topography) and PFO features (shunt size and presence of atrial septal aneurysm [ASA]) to classify patients into 3 validated categories of responsiveness to treatment with PFO closure. However, the distinctive associations of shunt size and ASA, alone and in combination, have not been completely delineated.

Objective

To evaluate the association of PFO closure with stroke recurrence according to shunt size and/or the presence of an ASA.

Design, Setting, and Participants

Pooled individual patient data from 6 randomized clinical trials conducted from February 2000 to October 2017 that compared PFO closure with medical therapy. Patients in North America, Europe, Australia, Brazil, and South Korea with PFO-associated stroke were included. Analysis was completed in January 2022.

Exposures

Transcatheter PFO closure plus antithrombotic therapy vs antithrombotic therapy alone, stratified into 4 groups based on the combination of 2 features: small vs large PFO shunt size and the presence or absence of an ASA.

Main Outcomes and Measures

Recurrent ischemic stroke.

Results

A total of 121 recurrent ischemic strokes occurred in the pooled 3740 patients (mean [SD] age, 45 [10] years; 1682 [45%] female) during a median (IQR) follow-up of 57 (23.7-63.8) months. Treatment with PFO closure was associated with reduced risk for recurrent ischemic stroke (adjusted hazard ratio [aHR], 0.41 [95% CI, 0.28-0.60]; P < .001). The reduction in hazard for recurrent stroke was greater for patients with both a large shunt and an ASA (aHR, 0.15 [95% CI, 0.06-0.35]) than for large shunt without ASA (aHR, 0.27 [95% CI, 0.14-0.56]), small shunt with ASA (aHR, 0.36 [95% CI, 0.17-0.78]), and small shunt without ASA (aHR, 0.68 [95% CI, 0.41-1.13]) (interaction P = .02). At 2 years, the absolute risk reduction of recurrent stroke was greater (5.5% [95% CI, 2.7-8.3]) in patients with large shunt and ASA than for patients in the other 3 categories (1.0% for all).

Conclusions and Relevance

Patients with both a large shunt and an ASA showed a substantially greater beneficial association with PFO closure than patients with large shunt alone, patients with small shunt and ASA, and patients with neither large shunt nor ASA. These findings, combined with other patient features, may inform shared patient-clinician decision-making.

Percutaneous Left Atrial Appendage Occlusion in Comparison to Non-Vitamin K Antagonist Oral Anticoagulant Among Patients With Atrial Fibrillation

Background This study aimed to compare percutaneous left atrial appendage occlusion (LAAO) with non-vitamin K antagonist oral anticoagulants among patients with atrial fibrillation. Methods and Results Using a US administrative database, 562 850 patients with atrial fibrillation were identified, among whom 8397 were treated with LAAO and 554 453 were treated with non-vitamin K antagonist oral anticoagulants between March 13, 2015 and December 31, 2018. Propensity score overlap weighting was used to balance baseline characteristics. The primary outcome was a composite end point of ischemic stroke or systemic embolism, major bleeding, and all-cause mortality. The mean age was 76.4±7.6 years; 280 097 (49.8%) were female. Mean follow-up was 1.5±1.0 years. LAAO was associated with no significant difference in the risk of the primary composite end point (hazard ratio [HR], 0.93 [0.84-1.03]), or the secondary outcomes including ischemic stroke/systemic embolism (HR, 1.07 [0.81-1.41]), and intracranial bleeding (HR, 1.08 [0.72-1.61]). LAAO was associated with a higher risk of major bleeding (HR, 1.22 [1.05-1.42], P=0.01) and a lower risk of mortality (HR, 0.73 [0.64-0.84], P<0.001). The lower risk of mortality associated with LAAO was most pronounced in patients with a prior history of intracranial bleeding. Conclusions In comparison to non-vitamin K antagonist oral anticoagulants, LAAO was associated with no significant difference in the risk of the composite outcome and a lower risk of mortality, which suggests LAAO might be a reasonable option in select patients with atrial fibrillation. The observation of higher bleeding risk associated with LAAO highlights the need to optimize postprocedural antithrombotic regimens as well as systematic efforts to assess and address bleeding predispositions.

Development of Parkinson Disease and Its Relationship with Incidentally Discovered White Matter Disease and Covert Brain Infarction in a Real-World Cohort

OBJECTIVE

This study aimed to examine the relationship between covert cerebrovascular disease, comprised of covert brain infarction and white matter disease, discovered incidentally in routine care, and subsequent Parkinson disease.

METHODS

Patients were ≥50 years and received neuroimaging for non-stroke indications in the Kaiser Permanente Southern California system from 2009 to 2019. Natural language processing identified incidentally discovered covert brain infarction and white matter disease and classified white matter disease severity. The Parkinson disease outcome was defined as 2 ICD diagnosis codes.

RESULTS

230,062 patients were included (median follow-up 3.72 years). A total of 1,941 Parkinson disease cases were identified (median time-to-event 2.35 years). Natural language processing identified covert cerebrovascular disease in 70,592 (30.7%) patients, 10,622 (4.6%) with covert brain infarction and 65,814 (28.6%) with white matter disease. After adjustment for known risk factors, white matter disease was associated with Parkinson disease (hazard ratio 1.67 [95%CI, 1.44, 1.93] for patients <70 years and 1.33 [1.18, 1.50] for those ≥70 years). Greater severity of white matter disease was associated with increased incidence of Parkinson disease(/1,000 person-years), from 1.52 (1.43, 1.61) in patients without white matter disease to 4.90 (3.86, 6.13) in those with severe disease. Findings were robust when more specific definitions of Parkinson disease were used. Covert brain infarction was not associated with Parkinson disease (adjusted hazard ratio = 1.05 [0.88, 1.24]).

INTERPRETATION

Incidentally discovered white matter disease was associated with subsequent Parkinson disease, an association strengthened with younger age and increased white matter disease severity. Incidentally discovered covert brain infarction did not appear to be associated with subsequent Parkinson disease. ANN NEUROL 2022;92:620-630.

Predicting Chronic Opioid Use Among Patients With Osteoarthritis Using Electronic Health Record Data

OBJECTIVE

To estimate the risk of a patient with osteoarthritis (OA) developing chronic opioid use (COU) within 1 year of a new opioid prescription by using electronic health record (EHR) data and predictive models.

METHODS

We used EHR data from 13 health care organizations to identify patients with OA with an opioid prescription between March 1, 2017 and February 28, 2019 and no record of opioid use in the prior 6 months. We evaluated 4 machine learning models to estimate patients' risk of COU (≥3 prescriptions ≥84 days, maximum gap ≤60 days). We also estimated the transportability of models to organizations outside the training set.

RESULTS

The cohort consisted of 33,894 patients with OA, of whom 2,925 (8.6%) developed COU within 1 year. All models demonstrated good discrimination, with the best-performing model (random forest) achieving an area under the receiver operating characteristic curve (AUC) of 0.728 (95% CI 0.711-0.745), but the simplest regression model performed nearly as well (AUC 0.717 [95% CI 0.699-0.734]). Predicted risk deciles spanned a range of 2% risk for the 10th percentile to 18% risk for the 90th percentile for well-calibrated models. Models showed highly variable discrimination across organizations (AUC 0.571-0.842).

CONCLUSIONS

We found that EHR-based predictive models could estimate the risk of future COU among patients with OA to help inform care decisions. Black-box methods did not have significant advantages over more interpretable models. Care should be taken when extending all models into organizations not included in model training because of a high variability in performance across held-out organizations.

Stratifying Future Stroke Risk with Incidentally Discovered White Matter Disease Severity and Covert Brain Infarct Site

BACKGROUND

Covert cerebrovascular disease (CCD) includes white matter disease (WMD) and covert brain infarction (CBI). Incidentally discovered CCD is associated with increased risk of subsequent symptomatic stroke. However, it is unknown whether the severity of WMD or the location of CBI predicts risk.

OBJECTIVES

The aim of this study was to examine the association of incidentally discovered WMD severity and CBI location with risk of subsequent symptomatic stroke.

METHOD

This retrospective cohort study includes patients aged ≥50 years old in the Kaiser Permanente Southern California health system who received neuroimaging for a nonstroke indication between 2009 and 2019. Incidental CBI and WMD were identified via natural language processing of the neuroimage report, and WMD severity was classified into grades.

RESULTS

A total of 261,960 patients received neuroimaging; 78,555 patients (30.0%) were identified to have incidental WMD and 12,857 patients (4.9%) to have incidental CBI. Increasing WMD severity is associated with an increased incidence rate of future stroke. However, the stroke incidence rate in CT-identified WMD is higher at each level of severity compared to rates in MRI-identified WMD. Patients with mild WMD via CT have a stroke incidence rate of 24.9 per 1,000 person-years, similar to that of patients with severe WMD via MRI. Among incidentally discovered CBI patients with a determined CBI location, 97.9% are subcortical rather than cortical infarcts. CBI confers a similar risk of future stroke, whether cortical or subcortical or whether MRI- or CT-detected.

CONCLUSIONS

Increasing severity of incidental WMD is associated with an increased risk of future symptomatic stroke, dependent on the imaging modality. Subcortical and cortical CBI conferred similar risks.

Risk Factors for Silent Brain Infarcts and White Matter Disease in a Real-World Cohort Identified by Natural Language Processing

OBJECTIVE

To assess the frequency of silent brain infarcts (SBIs) and white matter disease (WMD) and associations with stroke risk factors (RFs) in a real-world population.

PATIENTS AND METHODS

This was an observational study of patients 50 years or older in the Kaiser Permanente Southern California health system from January 1, 2009, through June 30, 2019, with head computed tomography or magnetic resonance imaging for nonstroke indications and no history of stroke, transient ischemic attack, or dementia. A natural language processing (NLP) algorithm was applied to the electronic health record to identify individuals with reported SBIs or WMD. Multivariable Poisson regression estimated risk ratios of demographic characteristics, RFs, and scan modality on the presence of SBIs or WMD.

RESULTS

Among 262,875 individuals, the NLP identified 13,154 (5.0%) with SBIs and 78,330 (29.8%) with WMD. Stroke RFs were highly prevalent. Advanced age was strongly associated with increased risk of SBIs (adjusted relative risks [aRRs], 1.90, 3.23, and 4.72 for those aged in their 60s, 70s, and ≥80s compared with those in their 50s) and increased risk of WMD (aRRs, 1.79, 3.02, and 4.53 for those aged in their 60s, 70s, and ≥80s compared with those in their 50s). Magnetic resonance imaging was associated with a reduced risk of SBIs (aRR, 0.87; 95% CI, 0.83 to 0.91) and an increased risk of WMD (aRR, 2.86; 95% CI, 2.83 to 2.90). Stroke RFs had modest associations with increased risk of SBIs or WMD.

CONCLUSION

An NLP algorithm can identify a large cohort of patients with incidentally discovered SBIs and WMD. Advanced age is strongly associated with incidentally discovered SBIs and WMD.

Does poor methodological quality of prediction modeling studies translate to poor model performance? An illustration in traumatic brain injury

BACKGROUND

Prediction modeling studies often have methodological limitations, which may compromise model performance in new patients and settings. We aimed to examine the relation between methodological quality of model development studies and their performance at external validation.

METHODS

We systematically searched for externally validated multivariable prediction models that predict functional outcome following moderate or severe traumatic brain injury. Risk of bias and applicability of development studies was assessed with the Prediction model Risk Of Bias Assessment Tool (PROBAST). Each model was rated for its presentation with sufficient detail to be used in practice. Model performance was described in terms of discrimination (AUC), and calibration. Delta AUC (dAUC) was calculated to quantify the percentage change in discrimination between development and validation for all models. Generalized estimation equations (GEE) were used to examine the relation between methodological quality and dAUC while controlling for clustering.

RESULTS

We included 54 publications, presenting ten development studies of 18 prediction models, and 52 external validation studies, including 245 unique validations. Two development studies (four models) were found to have low risk of bias (RoB). The other eight publications (14 models) showed high or unclear RoB. The median dAUC was positive in low RoB models (dAUC 8%, [IQR - 4% to 21%]) and negative in high RoB models (dAUC - 18%, [IQR - 43% to 2%]). The GEE showed a larger average negative change in discrimination for high RoB models (- 32% (95% CI: - 48 to - 15) and unclear RoB models (- 13% (95% CI: - 16 to - 10)) compared to that seen in low RoB models.

CONCLUSION

Lower methodological quality at model development associates with poorer model performance at external validation. Our findings emphasize the importance of adherence to methodological principles and reporting guidelines in prediction modeling studies.

External Validation of the FREEDOM Score for Individualized Decision Making Between CABG and PCI

BACKGROUND

Although randomized trials have established that coronary artery bypass grafting (CABG) is, on average, the most effective revascularization strategy compared with percutaneous coronary intervention (PCI) in patients with diabetes and multivessel disease (MVD), individual patients differ in many characteristics that can affect the benefits and harms of treatment. The FREEDOM (Future Revascularization Evaluation in Patients with Diabetes Mellitus) score was developed to predict different outcomes with CABG vs PCI on the basis of 8 patient characteristics and the smoking-treatment interaction.

OBJECTIVES

This study aimed to assess the ability of the 5-year major adverse cardiovascular event (MACE) model to predict treatment benefit of CABG vs PCI in the SYNTAX (Synergy Between Percutaneous Coronary Intervention With Taxus and Cardiac Surgery) and BEST (Bypass Surgery and Everolimus-Eluting Stent Implantation in the Treatment of Patients with Multivessel Coronary Artery Disease) trials.

METHODS

This study identified 702 patients with diabetes and MVD to mirror the FREEDOM participants. Discrimination was assessed by C-index, and calibration was assessed by calibration plots in the PCI and CABG arms, respectively. The ability of the FREEDOM score to predict treatment benefit of CABG vs PCI was assessed.

RESULTS

Overall, CABG was associated with a lower rate of 5-year MACE compared with PCI (12.4% vs 20.3%; log-rank P = 0.021) irrespective of a history of smoking (P_interaction = 0.975). Both discrimination and calibration were helpful in the PCI arm (C-index: 0.69; slope: 0.96, intercept: -0.24), but moderate in the CABG arm (C-index: 0.61; slope: 0.61; intercept: -0.53). The FREEDOM score showed some heterogeneity of treatment benefit.

CONCLUSIONS

The FREEDOM score could identify some heterogeneity of treatment benefit of CABG vs PCI for 5-year MACE. Until further prospective validations are performed, these results should be taken into consideration when using the FREEDOM score in patients with diabetes and MVD. (Synergy Between Percutaneous Coronary Intervention With Taxus and Cardiac Surgery [SYNTAX]; NCT00114972) (Bypass Surgery and Everolimus-Eluting Stent Implantation in the Treatment of Patients with Multivessel Coronary Artery Disease [BEST]; NCT00997828) (Future Revascularization Evaluation in Patients with Diabetes Mellitus [FREEDOM]; NCT00086450).

Generalizability of Cardiovascular Disease Clinical Prediction Models: 158 Independent External Validations of 104 Unique Models

Background:

While clinical prediction models (CPMs) are used increasingly commonly to guide patient care, the performance and clinical utility of these CPMs in new patient cohorts is poorly understood.

Methods:

We performed 158 external validations of 104 unique CPMs across 3 domains of cardiovascular disease (primary prevention, acute coronary syndrome, and heart failure). Validations were performed in publicly available clinical trial cohorts and model performance was assessed using measures of discrimination, calibration, and net benefit. To explore potential reasons for poor model performance, CPM-clinical trial cohort pairs were stratified based on relatedness, a domain-specific set of characteristics to qualitatively grade the similarity of derivation and validation patient populations. We also examined the model-based C-statistic to assess whether changes in discrimination were because of differences in case-mix between the derivation and validation samples. The impact of model updating on model performance was also assessed.

Results:

Discrimination decreased significantly between model derivation (0.76 [interquartile range 0.73–0.78]) and validation (0.64 [interquartile range 0.60–0.67], P<0.001), but approximately half of this decrease was because of narrower case-mix in the validation samples. CPMs had better discrimination when tested in related compared with distantly related trial cohorts. Calibration slope was also significantly higher in related trial cohorts (0.77 [interquartile range, 0.59–0.90]) than distantly related cohorts (0.59 [interquartile range 0.43–0.73], P=0.001). When considering the full range of possible decision thresholds between half and twice the outcome incidence, 91% of models had a risk of harm (net benefit below default strategy) at some threshold; this risk could be reduced substantially via updating model intercept, calibration slope, or complete re-estimation.

Conclusions:

There are significant decreases in model performance when applying cardiovascular disease CPMs to new patient populations, resulting in substantial risk of harm. Model updating can mitigate these risks. Care should be taken when using CPMs to guide clinical decision-making.

An Electronic Health Record-Compatible Model to Predict Personalized Treatment Effects From the Diabetes Prevention Program: A Cross-Evidence Synthesis Approach Using Clinical Trial and Real-World Data

OBJECTIVE

To develop an electronic health record (EHR)-based risk tool that provides point-of-care estimates of diabetes risk to support targeting interventions to patients most likely to benefit.

PATIENTS AND METHODS

A risk prediction model was developed and validated in a large observational database of patients with an index visit date between January 1, 2012, and December 31, 2016, with treatment effect estimates from risk-based reanalysis of clinical trial data. The risk model development cohort included 1.1 million patients with prediabetes from the OptumLabs Data Warehouse (OLDW); the validation cohort included a distinct sample of 1.1 million patients in OLDW. The randomly assigned clinical trial cohort included 3081 people from the Diabetes Prevention Program (DPP) study.

RESULTS

Eleven variables reliably obtainable from the EHR were used to predict diabetes risk. This model validated well in the OLDW (C statistic = 0.76; observed 3-year diabetes rate was 1.8% (95% confidence interval [CI], 1.7 to 1.9) in the lowest-risk quarter and 19.6% (19.4 to 19.8) in the highest-risk quarter). In the DPP, the hazard ratio (HR) for lifestyle modification was constant across all levels of risk (HR, 0.43; 95% CI, 0.35 to 0.53), whereas the HR for metformin was highly risk dependent (HR, 1.1; 95% CI, 0.61 to 2.0 in the lowest-risk quarter vs HR, 0.45; 95% CI, 0.35 to 0.59 in the highest-risk quarter). Fifty-three percent of the benefits of population-wide dissemination of the DPP lifestyle modification and 73% of the benefits of population-wide metformin therapy can be obtained by targeting the highest-risk quarter of patients.

CONCLUSION

The Tufts-Predictive Analytics and Comparative Effectiveness DPP Risk model is an EHR-compatible tool that might support targeted diabetes prevention to more efficiently realize the benefits of the DPP interventions.

Abstract TMP106: Pooled Analysis Of Five Randomized Trials Comparing Device Closure Of Patent Foramen Ovale After Stroke To Medical Treatment: Impact Of Residual Shunt On Stroke Outcomes

Introduction: Patent foramen ovale (PFO) closure is effective for secondary stroke prevention in well selected patients. After device implantation some patients have residual shunting. There was no impact of incomplete closure on recurrent stroke risk in any of the trials but outcome rates are low with little power to detect an effect.

Hypothesis: Patients with residual shunts after PFO closure have more recurrent strokes than those with complete elimination of shunting.

Methods: We pooled individual patient data from 5 of 6 randomized clinical trials comparing medical therapy to PFO closure + medical therapy. Residual shunt data were not available for 1 trial. This analysis only includes PFO closure subjects. Hazard ratios (HR) and 95% confidence intervals (CI) were estimated with Cox proportional hazards regression, adjusted for age, sex, coronary disease, diabetes, hypertension, hyperlipidemia, prior stroke/TIA, smoking, index event (stroke vs TIA), hypermobile septum, shunt size (large vs small) and infarct location (superficial vs deep). Multiple imputation of missing values for adjusted variables was used.

Results: Among 1475 patients treated with PFO closure and with non-missing residual shunt data, 149 (10%) had incomplete closure. The median time from procedure to shunt measurement was 204 days (25 th -75 th percentile, 181-722). In the complete closure group the primary endpoint of recurrent ischemic stroke occurred in 30 patients (2.3%; 95% CI 1.5-3.2%) compared to 4 (2.7%; 95% CI 0.7-6.7%) with incomplete closure. Unadjusted and adjusted HRs comparing incomplete vs complete closure were 1.37 (0.48, 3.92) and 1.69 (0.37, 4.95) respectively. The secondary endpoint of recurrent ischemic stroke, TIA, or vascular death occurred in 66 (5.0%; 95% CI 3.9-6.3%) patients with complete closure compared to 9 (6.0%; 95% CI 2.8-11.2%) without. Unadjusted and adjusted HRs were 1.26 (0.63, 2.54) and 1.39 (0.68, 2.83) respectively.

Conclusions: Our pooled analysis of individual subjects after PFO closure from 5 of 6 relevant randomized clinical trials, showed no association between residual shunt and recurrent stroke events. This analysis was limited by low rates of residual shunt and recurrent events even in this large, combined dataset.

Heterogeneity of Treatment Effects in an Analysis of Pooled Individual Patient Data From Randomized Trials of Device Closure of Patent Foramen Ovale After Stroke

Importance

Patent foramen ovale (PFO)-associated strokes comprise approximately 10% of ischemic strokes in adults aged 18 to 60 years. While device closure decreases stroke recurrence risk overall, the best treatment for any individual is often unclear.

Objective

To evaluate heterogeneity of treatment effect of PFO closure on stroke recurrence based on previously developed scoring systems.

Design, Setting, and Participants

Investigators for the Systematic, Collaborative, PFO Closure Evaluation (SCOPE) Consortium pooled individual patient data from all 6 randomized clinical trials that compared PFO closure plus medical therapy vs medical therapy alone in patients with PFO-associated stroke, and included a total of 3740 participants. The trials were conducted worldwide from 2000 to 2017.

Exposures

PFO closure plus medical therapy vs medical therapy alone. Subgroup analyses used the Risk of Paradoxical Embolism (RoPE) Score (a 10-point scoring system in which higher scores reflect younger age and the absence of vascular risk factors) and the PFO-Associated Stroke Causal Likelihood (PASCAL) Classification System, which combines the RoPE Score with high-risk PFO features (either an atrial septal aneurysm or a large-sized shunt) to classify patients into 3 categories of causal relatedness: unlikely, possible, and probable.

Main Outcomes and Measures

Ischemic stroke.

Results

Over a median follow-up of 57 months (IQR, 24-64), 121 outcomes occurred in 3740 patients. The annualized incidence of stroke with medical therapy was 1.09% (95% CI, 0.88%-1.36%) and with device closure was 0.47% (95% CI, 0.35%-0.65%) (adjusted hazard ratio [HR], 0.41 [95% CI, 0.28-0.60]). The subgroup analyses showed statistically significant interaction effects. Patients with low vs high RoPE Score had HRs of 0.61 (95% CI, 0.37-1.00) and 0.21 (95% CI, 0.11-0.42), respectively (P for interaction = .02). Patients classified as unlikely, possible, and probable using the PASCAL Classification System had HRs of 1.14 (95% CI, 0.53-2.46), 0.38 (95% CI, 0.22-0.65), and 0.10 (95% CI, 0.03-0.35), respectively (P for interaction = .003). The 2-year absolute risk reduction was -0.7% (95% CI, -4.0% to 2.6%), 2.1% (95% CI, 0.6%-3.6%), and 2.1% (95% CI, 0.9%-3.4%) in the unlikely, possible, and probable PASCAL categories, respectively. Device-associated adverse events were generally higher among patients classified as unlikely; the absolute risk increases in atrial fibrillation beyond day 45 after randomization with a device were 4.41% (95% CI, 1.02% to 7.80%), 1.53% (95% CI, 0.33% to 2.72%), and 0.65% (95% CI, -0.41% to 1.71%) in the unlikely, possible, and probable PASCAL categories, respectively.

Conclusions and Relevance

Among patients aged 18 to 60 years with PFO-associated stroke, risk reduction for recurrent stroke with device closure varied across groups classified by their probabilities that the stroke was causally related to the PFO. Application of this classification system has the potential to guide individualized decision-making.

External Validation of the SYNTAX Score II 2020

BACKGROUND

The SYNTAX score II 2020 (SSII-2020) was derived from cross correlation and externally validated in randomized trials to predict death and major adverse cardiac and cerebrovascular events (MACE) following percutaneous coronary intervention (PCI) and coronary artery bypass grafting (CABG) in patients with 3-vessel disease (3VD) and/or left main coronary artery disease (LMCAD).

OBJECTIVES

The authors aimed to investigate the SSII-2020's value in identifying the safest modality of revascularization in a non-randomized setting.

METHODS

Five-year mortality and MACE were assessed in 7,362 patients with 3VD and/or LMCAD enrolled in a Japanese PCI/CABG registry. The discriminative abilities of the SSII-2020 were assessed using Harrell's C statistic. Agreement between observed and predicted event rates following PCI or CABG and treatment benefit (absolute risk difference [ARD]) for these outcomes were assessed by calibration plots.

RESULTS

The SSII-2020 for 5-year mortality well predicted the prognosis after PCI and CABG (C-index = 0.72, intercept = -0.11, slope = 0.92). When patients were grouped according to the predicted 5-year mortality ARD, <4.5% (equipoise of PCI and CABG) and ≥4.5% (CABG better), the observed mortality rates after PCI and CABG were not significantly different in patients with lower predicted ARD (observed ARD: 2.1% [95% CI: -0.4% to 4.4%]), and the significant difference in survival in favor of CABG was observed in patients with higher predicted ARD (observed ARD: 9.7% [95% CI: 6.1%-13.3%]). For MACE, the SSII-2020 could not recommend a specific treatment with sufficient accuracy.

CONCLUSIONS

The SSII-2020 for predicting 5-year death has the potential to support decision making on revascularization in patients with 3VD and/or LMCAD.

COVID outcome prediction in the emergency department (COPE): using retrospective Dutch hospital data to develop simple and valid models for predicting mortality and need for intensive care unit admission in patients who present at the emergency department with suspected COVID-19

OBJECTIVES

Develop simple and valid models for predicting mortality and need for intensive care unit (ICU) admission in patients who present at the emergency department (ED) with suspected COVID-19.

DESIGN

Retrospective.

SETTING

Secondary care in four large Dutch hospitals.

PARTICIPANTS

Patients who presented at the ED and were admitted to hospital with suspected COVID-19. We used 5831 first-wave patients who presented between March and August 2020 for model development and 3252 second-wave patients who presented between September and December 2020 for model validation.

OUTCOME MEASURES

We developed separate logistic regression models for in-hospital death and for need for ICU admission, both within 28 days after hospital admission. Based on prior literature, we considered quickly and objectively obtainable patient characteristics, vital parameters and blood test values as predictors. We assessed model performance by the area under the receiver operating characteristic curve (AUC) and by calibration plots.

RESULTS

Of 5831 first-wave patients, 629 (10.8%) died within 28 days after admission. ICU admission was fully recorded for 2633 first-wave patients in 2 hospitals, with 214 (8.1%) ICU admissions within 28 days. A simple model-COVID outcome prediction in the emergency department (COPE)-with age, respiratory rate, C reactive protein, lactate dehydrogenase, albumin and urea captured most of the ability to predict death. COPE was well calibrated and showed good discrimination for mortality in second-wave patients (AUC in four hospitals: 0.82 (95% CI 0.78 to 0.86); 0.82 (95% CI 0.74 to 0.90); 0.79 (95% CI 0.70 to 0.88); 0.83 (95% CI 0.79 to 0.86)). COPE was also able to identify patients at high risk of needing ICU admission in second-wave patients (AUC in two hospitals: 0.84 (95% CI 0.78 to 0.90); 0.81 (95% CI 0.66 to 0.95)).

CONCLUSIONS

COPE is a simple tool that is well able to predict mortality and need for ICU admission in patients who present to the ED with suspected COVID-19 and may help patients and doctors in decision making.

Association of Silent Cerebrovascular Disease Identified Using Natural Language Processing and Future Ischemic Stroke

BACKGROUND AND OBJECTIVE

Silent cerebrovascular disease (SCD), comprised of silent brain infarction (SBI) and white matter disease (WMD), is commonly found incidentally on neuroimaging scans obtained in routine clinical care. However, their prognostic significance is not known. We aimed to estimate the incidence of, and risk increase in, future stroke in patients with incidentally-discovered SCD. METHODS: Patients in Kaiser Permanente Southern California (KPSC) health system aged ≥ 50, without prior ischemic stroke, transient ischemic attack, or dementia/Alzheimer's disease receiving a head CT or MRI between 2009-2019 were included. SBI and WMD were identified by natural language processing (NLP) from the neuroimage report.

RESULTS

Among 262,875 individuals receiving neuroimaging, NLP identified 13,154 (5.0%) with SBI and 78,330 (29.8%) with WMD. The incidence of future stroke was 32.5 (95% CI 31.1, 33·9) per 1,000 patient-years for patients with SBI; 1.·3 (95% CI 18.9, 19.8) for patients with WMD and 6.8 (95% CI 6.7, 7.0) for patients without SCD. The crude HR associated with SBI was 3.40 (95% CI 3.25 to 3.56); and for WMD was 2.63 (95% CI 2.54 to 2·71). With MRI-discovered SBI, the adjusted HR was 2.95 (95% CI 2.53 to 3.44) for those < age 65 and 2.15 (95% CI 1.91 to 2.41) for those ≥ age 65. With CT scan, the adjusted HR was 2.48 (95% CI 2.19 to 2.81) for those < age 65 and 1.81 (95% CI 1.71 to 1.91) for those >= age 65. The adjusted HR associated with a finding of WMD was 1.76 (95% CI 1.69 to 1.82) and was not modified by age or imaging modality.

DISCUSSION

Incidentally-discovered SBI and WMD are common and associated with increased risk of subsequent symptomatic stroke representing an important opportunity for stroke prevention.

External Validations of Cardiovascular Clinical Prediction Models: A Large-Scale Review of the Literature

BACKGROUND

There are many clinical prediction models (CPMs) available to inform treatment decisions for patients with cardiovascular disease. However, the extent to which they have been externally tested, and how well they generally perform has not been broadly evaluated.

METHODS

A SCOPUS citation search was run on March 22, 2017 to identify external validations of cardiovascular CPMs in the Tufts Predictive Analytics and Comparative Effectiveness CPM Registry. We assessed the extent of external validation, performance heterogeneity across databases, and explored factors associated with model performance, including a global assessment of the clinical relatedness between the derivation and validation data.

RESULTS

We identified 2030 external validations of 1382 CPMs. Eight hundred seven (58%) of the CPMs in the Registry have never been externally validated. On average, there were 1.5 validations per CPM (range, 0-94). The median external validation area under the receiver operating characteristic curve was 0.73 (25th-75th percentile [interquartile range (IQR)], 0.66-0.79), representing a median percent decrease in discrimination of -11.1% (IQR, -32.4% to +2.7%) compared with performance on derivation data. 81% (n=1333) of validations reporting area under the receiver operating characteristic curve showed discrimination below that reported in the derivation dataset. 53% (n=983) of the validations report some measure of CPM calibration. For CPMs evaluated more than once, there was typically a large range of performance. Of 1702 validations classified by relatedness, the percent change in discrimination was -3.7% (IQR, -13.2 to 3.1) for closely related validations (n=123), -9.0 (IQR, -27.6 to 3.9) for related validations (n=862), and -17.2% (IQR, -42.3 to 0) for distantly related validations (n=717; P<0.001).

CONCLUSIONS

Many published cardiovascular CPMs have never been externally validated, and for those that have, apparent performance during development is often overly optimistic. A single external validation appears insufficient to broadly understand the performance heterogeneity across different settings.

Large-scale validation of the prediction model risk of bias assessment Tool (PROBAST) using a short form: high risk of bias models show poorer discrimination

OBJECTIVE

To assess whether the Prediction model Risk Of Bias ASsessment Tool (PROBAST) and a shorter version of this tool can identify clinical prediction models (CPMs) that perform poorly at external validation.

STUDY DESIGN AND SETTING

We evaluated risk of bias (ROB) on 102 CPMs from the Tufts CPM Registry, comparing PROBAST to a short form consisting of six PROBAST items anticipated to best identify high ROB. We then applied the short form to all CPMs in the Registry with at least 1 validation (n=556) and assessed the change in discrimination (dAUC) in external validation cohorts (n=1,147).

RESULTS

PROBAST classified 98/102 CPMS as high ROB. The short form identified 96 of these 98 as high ROB (98% sensitivity), with perfect specificity. In the full CPM registry, 527 of 556 CPMs (95%) were classified as high ROB, 20 (3.6%) low ROB, and 9 (1.6%) unclear ROB. Only one model with unclear ROB was reclassified to high ROB after full PROBAST assessment of all low and unclear ROB models. Median change in discrimination was significantly smaller in low ROB models (dAUC -0.9%, IQR -6.2-4.2%) compared to high ROB models (dAUC -11.7%, IQR -33.3-2.6%; P<0.001).

CONCLUSION

High ROB is pervasive among published CPMs. It is associated with poor discriminative performance at validation, supporting the application of PROBAST or a shorter version in CPM reviews.

Targeting of the diabetes prevention program leads to substantial benefits when capacity is constrained

OBJECTIVE

Approximately 84 million people in the USA have pre-diabetes, but only a fraction of them receive proven effective therapies to prevent type 2 diabetes. We estimated the value of prioritizing individuals at highest risk of progression to diabetes for treatment, compared to non-targeted treatment of individuals meeting inclusion criteria for the Diabetes Prevention Program (DPP).

METHODS

Using microsimulation to project outcomes in the DPP trial population, we compared two interventions to usual care: (1) lifestyle modification and (2) metformin administration. For each intervention, we compared targeted and non-targeted strategies, assuming either limited or unlimited program capacity. We modeled the individualized risk of developing diabetes and projected diabetic outcomes to yield lifetime costs and quality-adjusted life expectancy, from which we estimated net monetary benefits (NMB) for both lifestyle and metformin versus usual care.

RESULTS

Compared to usual care, lifestyle modification conferred positive benefits and reduced lifetime costs for all eligible individuals. Metformin's NMB was negative for the lowest population risk quintile. By avoiding use when costs outweighed benefits, targeted administration of metformin conferred a benefit of $500 per person. If only 20% of the population could receive treatment, when prioritizing individuals based on diabetes risk, rather than treating a 20% random sample, the difference in NMB ranged from $14,000 to $20,000 per person.

CONCLUSIONS

Targeting active diabetes prevention to patients at highest risk could improve health outcomes and reduce costs compared to providing the same intervention to a similar number of patients with pre-diabetes without targeted selection.

Agreement between neuroimages and reports for natural language processing-based detection of silent brain infarcts and white matter disease

BACKGROUND

There are numerous barriers to identifying patients with silent brain infarcts (SBIs) and white matter disease (WMD) in routine clinical care. A natural language processing (NLP) algorithm may identify patients from neuroimaging reports, but it is unclear if these reports contain reliable information on these findings.

METHODS

Four radiology residents reviewed 1000 neuroimaging reports (RI) of patients age > 50 years without clinical histories of stroke, TIA, or dementia for the presence, acuity, and location of SBIs, and the presence and severity of WMD. Four neuroradiologists directly reviewed a subsample of 182 images (DR). An NLP algorithm was developed to identify findings in reports. We assessed interrater reliability for DR and RI, and agreement between these two and with NLP.

RESULTS

For DR, interrater reliability was moderate for the presence of SBIs (k = 0.58, 95 % CI 0.46-0.69) and WMD (k = 0.49, 95 % CI 0.35-0.63), and moderate to substantial for characteristics of SBI and WMD. Agreement between DR and RI was substantial for the presence of SBIs and WMD, and fair to substantial for characteristics of SBIs and WMD. Agreement between NLP and DR was substantial for the presence of SBIs (k = 0.64, 95 % CI 0.53-0.76) and moderate (k = 0.52, 95 % CI 0.39-0.65) for the presence of WMD.

CONCLUSIONS

Neuroimaging reports in routine care capture the presence of SBIs and WMD. An NLP can identify these findings (comparable to direct imaging review) and can likely be used for cohort identification.

The cumulative incidence of dysphagia and dysphagia-free survival in persons diagnosed with amyotrophic lateral sclerosis

INTRODUCTION/AIMS

Dysphagia worsens mortality and quality of life for persons diagnosed with amyotrophic lateral sclerosis (ALS), yet our understanding of its incidence and timing remains limited. In this study we sought to estimate dysphagia incidence and dysphagia-free survival over time.

METHODS

Using data from the Pooled Resource Open-Access ALS Clinical Trials Database, we compared characteristics of persons with and without dysphagia upon study entry. To account for competing mortality risk, we used Kaplan-Meier curves to estimate the cumulative incidence of dysphagia and the median number of days until the development of dysphagia or death in those without dysphagia at study entry.

RESULTS

Patients with dysphagia upon study entry were more likely to have bulbar onset and had faster rates of functional decline and shorter diagnostic delays. The cumulative incidence of new-onset dysphagia was 44% at 1 year and 64% at 2 years after trial enrollment for those with spinal onset, and 85% and 92% for those with bulbar onset. The median duration of dysphagia-free survival after trial enrollment was 11.5 months for those with spinal onset and 3.2 months for those with bulbar onset.

DISCUSSION

Our findings underscore the high risk for dysphagia development and support the need for early dysphagia referral and evaluation to minimize the risk of serious dysphagia-related complications.

Abstract P595: Incidentally-Discovered Asymptomatic Ischemic Lesions Identified Using Natural Language Processing Are Strong Risk Factors for Dementia

Background: White matter disease (WMD) and silent brain infarction (SBI) are known to be risk markers for dementia. Nevertheless, the effects of incidentally-discovered WMD and SBI on routinely-obtained neuroimages is unknown.

Methods: In the retrospective cohort study, Kaiser Permanente-Southern California (KPSC) health plan enrollees aged ≥ 50 years old with a brain cross-sectional image (CT or MRI) between 2009-2019 and without a prior history of ischemic stroke, transient ischemic attack, or dementia were identified. Natural language processing (NLP) was used to identify patients with SBI and WMD on the first qualifying neuroimaging report. Cox proportional hazards regression model was applied to estimate the effects of WMD and SBI on dementia, controlling for major cardiovascular risk factors.

Results: Among 262,875 individuals receiving brain neuroimaging, 13,154 (5.0%) and 78,330 (29.8%) had SBI and WMD, respectively. The table below summarizes the crude dementia incidence rates. The crude hazard ratio (HR) was 3.69 (95% CI 3.58-3.80) for WMD and 2.80 (95% CI 2.68-2.93) for SBI. In the multivariable model controlling for all major cardiovascular risk factors, the effect of WMD on dementia risk was found to be stronger in younger compared to older patients and for CT- compared to MRI-discovered lesions. With CT, the adjusted HR was 3.08 (95% CI 2.84-3.34) for those < age 70 and 1.90 (95% CI 1.83-1.97) for those ≥ age 70. With MRI, the adjusted HR was 2.58 (95% CI 2.31-2.87) for those < age 70 and 1.59 (95% CI 1.45-1.75) for those ≥ age 70. The adjusted HR associated with a finding of SBI was 2.01 (95% CI 1.78-2.32) for patients age <70 and 1.26 (95% CI 1.20-1.32) for patients age ≥70, and was not modified by imaging modality.

Conclusion: Incidentally-discovered WMD and SBI are common in patients ≥ age 50 and are associated with substantial increase in the risk of subsequent dementia. The findings may represent an opportunity for dementia prevention.

Abstract P57: Risk Factors for Incidentally Discovered Silent Brain Infarcts and White Matter Disease in a Real World Cohort Identified by Artificial Intelligence

Introduction: Silent brain infarcts (SBIs) and white matter disease (WMD) are highly prevalent and associated with increased risk of ischemic stroke in patients with traditional stroke risk factors (RFs) in prospective cohort studies. Their frequency and associations with stroke RFs have not been well described in real world populations.

Methods: This was a cross-sectional study of patients age ≥ 50 in the Kaiser Permanente-Southern California (KPSC) health system between 2009-2019 with a head CT or MRI for non-stroke indications and no history of ischemic stroke, transient ischemic attack, or dementia. A natural language processing (NLP) algorithm developed at Mayo Clinic and Tufts Medical Center was applied to the KPSC EHR to identify individuals with reported SBIs or WMD. Multivariable Poisson regression with robust error variance was used to estimate risk ratios of demographics, stroke RFs (from the Framingham Stroke Risk Score), and scan modality on the presence of SBIs or WMD.

Results: Among 262,875 individuals, the NLP identified 13,154 (5.0%) with SBIs and 78,330 (29.8%) with WMD. Stroke RFs were highly prevalent in this cohort. The majority underwent CTs (74.8%) instead of MRIs as their initial neuroimaging. After adjustment for demographics and RFs, advanced age demonstrated a strong association with increased risk of SBIs and WMD (table). MRI was associated with a reduced risk of reported SBIs (ARR: 0.87, 95% CI 0.83-0.91) and an increased risk of reported WMD (ARR 2.86, 95% CI 2.83-2.90). Despite being prevalent, traditional stroke RFs had weak associations with increased risk of SBIs or increased risk of WMD.

Conclusions: Advanced age is strongly associated with incidentally discovered SBIs and WMD on neuroimaging studies obtained in routine care. The development of SBIs and WMD may not be fully attributable to traditional stroke RFs.

Abstract P604: Incidentally-Discovered Asymptomatic Ischemic Lesions Identified Using Natural Language Processing Are Strong Risk Markers for Future Ischemic Stroke

Background: White matter disease (WMD) and silent brain infarction (SBI) are known to be risk markers for stroke. Nevertheless, the predictive value of these changes when seen incidentally on routinely-obtained neuroimages is unknown.

Methods: In this retrospective cohort study, Kaiser Permanente-Southern California health plan enrollees aged ≥ 50 years old with a brain CT or MRI scan between 2009-2019 and without a prior history of ischemic stroke, transient ischemic attack, or dementia were identified. Natural language processing (NLP) was used to identify patients with SBI and WMD on the index neuroimaging report. We used Cox proportional hazards to estimate the risk of future ischemic stroke associated with the presence of SBI and of WMD, controlling for major stroke risk factors.

Results: Among 262,875 individuals receiving brain neuroimaging, 13,154 (5.0%) and 78,330 (29.8%) had SBI and WMD, respectively. The Table below summarizes the crude stroke incidence rates. The crude hazard ratio (HR) was 3.40 (95% CI 3.25-3.56) for SBI and 2.63 (95% CI 2.54-2.71) for WMD. In the multivariable model controlling for all major stroke risk factors, the effect of SBI was found to be stronger in younger versus older patients and for MRI- versus CT-discovered lesions. With MRI, the average adjusted HR over time was 2.95 (95% CI 2.53-3.44) for those < age 65 and 2.15 (95% CI 1.91-2.41) for those ≥ age 65. With CT scan, the average adjusted HR over time was 2.48 (95%CI 2.19-2.81) for those < age 65 and 1.81 (95% CI 1.71-1.91) for those ≥ age 65. The adjusted HR associated with a finding of WMD was 1.76 (95% CI 1.69-1.82) and was not modified by age or imaging modality. The effect of SBI decreased gradually over time, while the effect of WMD remained constant.

Conclusion: Incidentally-discovered SBI and WMD are common in patients ≥ age 50 and are associated with substantial increases in the risk of subsequent symptomatic stroke. The findings may represent an opportunity for stroke prevention.

Proposal for Updated Nomenclature and Classification of Potential Causative Mechanism in Patent Foramen Ovale-Associated Stroke

Importance

Recent epidemiologic and therapeutic advances have transformed understanding of the role of and therapeutic approach to patent foramen ovale (PFO) in ischemic stroke. Patent foramen ovale is likely responsible for approximately 5% of all ischemic strokes and 10% of those occurring in young and middle-aged adults.

Observations

Randomized clinical trials have demonstrated that, to prevent recurrent ischemic stroke in patients with PFO and an otherwise-cryptogenic index ischemic stroke, PFO closure is superior to antiplatelet medical therapy alone; these trials have provided some evidence that, among medical therapy options, anticoagulants may be more effective than antiplatelet agents.

Conclusions and Relevance

These new data indicate a need to update classification schemes of causative mechanisms in stroke, developed in an era in which an association between PFO and stroke was viewed as uncertain. We propose a revised general nomenclature and classification framework for PFO-associated stroke and detailed revisions for the 3 major stroke subtyping algorithms in wide use.

Precision Health Analytics With Predictive Analytics and Implementation Research: JACC State-of-the-Art Review

Emerging data science techniques of predictive analytics expand the quality and quantity of complex data relevant to human health and provide opportunities for understanding and control of conditions such as heart, lung, blood, and sleep disorders. To realize these opportunities, the information sources, the data science tools that use the information, and the application of resulting analytics to health and health care issues will require implementation research methods to define benefits, harms, reach, and sustainability; and to understand related resource utilization implications to inform policymakers. This JACC State-of-the-Art Review is based on a workshop convened by the National Heart, Lung, and Blood Institute to explore predictive analytics in the context of implementation science. It highlights precision medicine and precision public health as complementary and compelling applications of predictive analytics, and addresses future research and training endeavors that might further foster the application of predictive analytics in clinical medicine and public health.