Re-estimation improved the performance of two Framingham cardiovascular risk equations and the Pooled Cohort equations: A nationwide registry analysis

Improving Cardiovascular Disease Primary Prevention Treatment Thresholds in a New England Health Care System

Background

Atherosclerotic cardiovascular disease (ASCVD) risk estimation based on the pooled cohort equation (PCE) overestimates in population-based cohorts. Whether it performs equally across disaggregated demographics in health care populations is less known.

Objectives

The purpose of the study was to recalibrate PCE and rederive prevention thresholds in a contemporary health care system and evaluate its performance across sociodemographics.

Methods

We retrospectively inspected electronic health records between 2010 to 2012 and 2020 to 2022 within Mass General Brigham health care in New England region. We compared performance of the original vs recalibrated PCE measured by calibration, discrimination, reclassification rate, and net benefit among 160,926 patients aged 40 to 79 years and without prior ASCVD or lipid-lowering medication.

Results

Of the 160,926 patients (mean age: 54.6 ± 8.6 years; 61.4% female), 20,373 (12.7%) developed ASCVD over 10 years. The original PCE globally underestimated ASCVD risk (observed vs predicted incidence rate: 0.13 vs 0.05). Recalibration upclassified risk primarily among individuals with low-to-borderline risk by the original PCE and additionally identified 40% of patients who had undergone ASCVD events yet deemed statin-ineligible based on the original PCE. Treatment thresholds yielding the greatest net benefit were ≥24.0% for women (+23.3%) vs ≥26.0% for men (+18.7%), whereas ≥26.0% for White or other race (+24.7%) vs ≥14.0% Black or African American (+12.5%), respectively. Specifically, Hispanic or Latino and non-Hispanic Black patients conferred the greatest sensitivity improvement at ≥12.3% threshold compared to higher ≥23.6% among non-Hispanic Asian or Pacific Islanders. Generally, lower thresholds earlier in life were optimal.

Conclusions

Recalibration and personalized treatment thresholds derived within a health system may improve prevention treatment allocation efficiency.

Nutritional and Lifestyle Features in a Mediterranean Cohort: An Epidemiological Instrument for Categorizing Metabotypes Based on a Computational Algorithm

Background and Objectives: Modern classification and categorization of individuals' health requires personalized variables such as nutrition, physical activity, lifestyle, and medical data through advanced analysis and clustering methods involving machine learning tools. The objective of this project was to categorize Mediterranean dwellers' health factors and design metabotypes to provide personalized well-being in order to develop professional implementation tools in addition to characterizing nutritional and lifestyle features in such populations. Materials and Methods: A two-phase observational study was conducted by the Pharmacists Council to identify Spanish nutritional and lifestyle characteristics. Adults over 18 years of age completed questionnaires on general lifestyle habits, dietary patterns (FFQ, MEDAS-17 p), physical activity (IPAQ), quality of life (SF-12), and validated well-being indices (LS7, MEDLIFE, HHS, MHL). Subsequently, exploratory factor, clustering, and random forest analysis methods were conducted to objectively define the metabotypes considering population determinants. Results: A total of 46.4% of the sample (n = 5496) had moderate-to-high adherence to the Mediterranean diet (>8 points), while 71% of the participants declared that they had moderate physical activity. Almost half of the volunteers had a good self-perception of health (49.9%). Regarding lifestyle index, population LS7 showed a fair cardiovascular health status (7.9 ± 1.7), as well as moderate quality of life by MEDLIFE (9.3 ± 2.6) and MHL scores (2.4 ± 0.8). In addition, five metabotype models were developed based on 26 variables: Westernized Millennial (28.6%), healthy (25.1%), active Mediterranean (16.5%), dysmetabolic/pre-morbid (11.5%), and metabolically vulnerable/pro-morbid (18.3%). Conclusions: The support of tools related to precision nutrition and lifestyle integrates well-being characteristics and contributes to reducing the impact of unhealthy lifestyle habits with practical implications for primary care. Combining lifestyle, metabolic, and quality of life traits will facilitate personalized precision interventions and the implementation of targeted public health policies.

Long-term cardiovascular risks and the impact of statin treatment on socioeconomic inequalities: a microsimulation model

BACKGROUND

UK cardiovascular disease (CVD) incidence and mortality have declined in recent decades but socioeconomic inequalities persist.

AIM

To present a new CVD model, and project health outcomes and the impact of guideline-recommended statin treatment across quintiles of socioeconomic deprivation in the UK.

DESIGN AND SETTING

A lifetime microsimulation model was developed using 117 896 participants in 16 statin trials, 501 854 UK Biobank (UKB) participants, and quality-of-life data from national health surveys.

METHOD

A CVD microsimulation model was developed using risk equations for myocardial infarction, stroke, coronary revascularisation, cancer, and vascular and non-vascular death, estimated using trial data. The authors calibrated and further developed this model in the UKB cohort, including further characteristics and a diabetes risk equation, and validated the model in UKB and Whitehall II cohorts. The model was used to predict CVD incidence, life expectancy, quality-adjusted life years (QALYs), and the impact of UK guideline-recommended statin treatment across socioeconomic deprivation quintiles.

RESULTS

Age, sex, socioeconomic deprivation, smoking, hypertension, diabetes, and cardiovascular events were key CVD risk determinants. Model-predicted event rates corresponded well to observed rates across participant categories. The model projected strong gradients in remaining life expectancy, with 4-5-year (5-8 QALYs) gaps between the least and most socioeconomically deprived quintiles. Guideline-recommended statin treatment was projected to increase QALYs, with larger gains in quintiles of higher deprivation.

CONCLUSION

The study demonstrated the potential of guideline-recommended statin treatment to reduce socioeconomic inequalities. This CVD model is a novel resource for individualised long-term projections of health outcomes of CVD treatments.

Incorporating longitudinal history of risk factors into atherosclerotic cardiovascular disease risk prediction using deep learning

It is increasingly clear that longitudinal risk factor levels and trajectories are related to risk for atherosclerotic cardiovascular disease (ASCVD) above and beyond single measures. Currently used in clinical care, the Pooled Cohort Equations (PCE) are based on regression methods that predict ASCVD risk based on cross-sectional risk factor levels. Deep learning (DL) models have been developed to incorporate longitudinal data for risk prediction but its benefit for ASCVD risk prediction relative to the traditional Pooled Cohort Equations (PCE) remain unknown. Our study included 15,565 participants from four cardiovascular disease cohorts free of baseline ASCVD who were followed for adjudicated ASCVD. Ten-year ASCVD risk was calculated in the training set using our benchmark, the PCE, and a longitudinal DL model, Dynamic-DeepHit. Predictors included those incorporated in the PCE: sex, race, age, total cholesterol, high density lipid cholesterol, systolic and diastolic blood pressure, diabetes, hypertension treatment and smoking. The discrimination and calibration performance of the two models were evaluated in an overall hold-out testing dataset. Of the 15,565 participants in our dataset, 2170 (13.9%) developed ASCVD. The performance of the longitudinal DL model that incorporated 8 years of longitudinal risk factor data improved upon that of the PCE [AUROC: 0.815 (CI 0.782-0.844) vs 0.792 (CI 0.760-0.825)] and the net reclassification index was 0.385. The brier score for the DL model was 0.0514 compared with 0.0542 in the PCE. Incorporating longitudinal risk factors in ASCVD risk prediction using DL can improve model discrimination and calibration.

Incorporating longitudinal history of risk factors into atherosclerotic cardiovascular disease risk prediction using deep learning

Background

It is increasingly clear that longitudinal risk factor levels and trajectories are related to risk for atherosclerotic cardiovascular disease (ASCVD) above and beyond single measures. Currently used in clinical care, the Pooled Cohort Equations (PCE) are based on regression methods that predict ASCVD risk based on cross-sectional risk factor levels. Deep learning (DL) models have been developed to incorporate longitudinal data for risk prediction but its benefit for ASCVD risk prediction relative to the traditional Pooled Cohort Equations (PCE) remain unknown.

Objective

To develop a ASCVD risk prediction model that incorporates longitudinal risk factors using deep learning.

Methods

Our study included 15,565 participants from four cardiovascular disease cohorts free of baseline ASCVD who were followed for adjudicated ASCVD. Ten-year ASCVD risk was calculated in the training set using our benchmark, the PCE, and a longitudinal DL model, Dynamic-DeepHit. Predictors included those incorporated in the PCE: sex, race, age, total cholesterol, high density lipid cholesterol, systolic and diastolic blood pressure, diabetes, hypertension treatment and smoking. The discrimination and calibration performance of the two models were evaluated in an overall hold-out testing dataset.

Results

Of the 15,565 participants in our dataset, 2,170 (13.9%) developed ASCVD. The performance of the longitudinal DL model that incorporated 8 years of longitudinal risk factor data improved upon that of the PCE [AUROC: 0.815 (CI: 0.782-0.844) vs 0.792 (CI: 0.760-0.825)] and the net reclassification index was 0.385. The brier score for the DL model was 0.0514 compared with 0.0542 in the PCE.

Conclusion

Incorporating longitudinal risk factors in ASCVD risk prediction using DL can improve model discrimination and calibration.

Clinical utility gains from incorporating comorbidity and geographic location information into risk estimation equations for atherosclerotic cardiovascular disease

OBJECTIVE

There are over 363 customized risk models of the American College of Cardiology and the American Heart Association (ACC/AHA) pooled cohort equations (PCE) in the literature, but their gains in clinical utility are rarely evaluated. We build new risk models for patients with specific comorbidities and geographic locations and evaluate whether performance improvements translate to gains in clinical utility.

MATERIALS AND METHODS

We retrain a baseline PCE using the ACC/AHA PCE variables and revise it to incorporate subject-level information of geographic location and 2 comorbidity conditions. We apply fixed effects, random effects, and extreme gradient boosting (XGB) models to handle the correlation and heterogeneity induced by locations. Models are trained using 2 464 522 claims records from Optum©'s Clinformatics® Data Mart and validated in the hold-out set (N = 1 056 224). We evaluate models' performance overall and across subgroups defined by the presence or absence of chronic kidney disease (CKD) or rheumatoid arthritis (RA) and geographic locations. We evaluate models' expected utility using net benefit and models' statistical properties using several discrimination and calibration metrics.

RESULTS

The revised fixed effects and XGB models yielded improved discrimination, compared to baseline PCE, overall and in all comorbidity subgroups. XGB improved calibration for the subgroups with CKD or RA. However, the gains in net benefit are negligible, especially under low exchange rates.

CONCLUSIONS

Common approaches to revising risk calculators incorporating extra information or applying flexible models may enhance statistical performance; however, such improvement does not necessarily translate to higher clinical utility. Thus, we recommend future works to quantify the consequences of using risk calculators to guide clinical decisions.

Patient-oriented risk score for predicting death 1 year after myocardial infarction: the SweDen risk score

OBJECTIVES

Our aim was to derive, based on the SWEDEHEART registry, and validate, using the Western Denmark Heart registry, a patient-oriented risk score, the SweDen score, which could calculate the risk of 1-year mortality following a myocardial infarction (MI).

METHODS

The factors included in the SweDen score were age, sex, smoking, diabetes, heart failure and statin use. These were chosen a priori by the SWEDEHEART steering group based on the premise that the factors were information known by the patients themselves. The score was evaluated using various statistical methods such as time-dependent receiver operating characteristics curves of the linear predictor, area under the curve metrics, Kaplan-Meier survivor curves and the calibration slope.

RESULTS

The area under the curve values were 0.81 in the derivation data and 0.76 in the validation data. The Kaplan-Meier curves showed similar patient profiles across datasets. The calibration slope was 1.03 (95% CI 0.99 to 1.08) in the validation data using the linear predictor from the derivation data.

CONCLUSIONS

The SweDen risk score is a novel tool created for patient use. The risk score calculator will be available online and presents mortality risk on a colour scale to simplify interpretation and to avoid exact life span expectancies. It provides a validated patient-oriented risk score predicting the risk of death within 1 year after suffering an MI, which visualises the benefit of statin use and smoking cessation in a simple way.

Updating Framingham CVD risk score using waist circumference and estimated cardiopulmonary function: a cohort study based on a southern Xinjiang population

Purpose

To explore the association between waist circumference (WC), estimated cardiopulmonary function (eCRF), and cardiovascular disease (CVD) risk in southern Xinjiang. Update the Framingham model to make it more suitable for the southern Xinjiang population.

Methods

Data were collected from 7705 subjects aged 30–74 years old in Tumushuke City, the 51st Regiment of Xinjiang Production and Construction Corps. CVD was defined as an individual's first diagnosis of non-fatal acute myocardial infarction, death from coronary heart disease, and fatal or non-fatal stroke. The Cox proportional hazards regression analysis was used to analyze the association between WC, eCRF and CVD risk. Restricted cubic spline plots were drawn to describe the association of the two indicators with CVD risk. We update the model by incorporating the new variables into the Framingham model and re-estimating the coefficients. The discrimination of the model is evaluated using AUC, NRI, and IDI metrics. Model calibration is evaluated using pseudo R² values.

Results

WC was an independent risk factor for CVD (multivariate HR: 1.603 (1.323, 1.942)), eCRF was an independent protective factor for CVD (multivariate HR: 0.499 (0.369, 0.674)). There was a nonlinear relationship between WC and CVD risk (nonlinear χ2 = 12.43, P = 0.002). There was a linear association between eCRF and CVD risk (non-linear χ2 = 0.27, P = 0.6027). In the male, the best risk prediction effect was obtained when WC and eCRF were added to the model (AUC = 0.763((0.734,0.792)); pseudo R² = 0.069). In the female, the best risk prediction effect was obtained by adding eCRF to the model (AUC = 0.757 (0.734,0.779); pseudo R² = 0.107).

Conclusion

In southern Xinjiang, WC is an independent risk factor for CVD. eCRF is an independent protective factor for CVD. We recommended adding WC and eCRF in the male model and only eCRF in the female model for better risk prediction.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12889-022-14110-y.

The roles of predictors in cardiovascular risk models - a question of modeling culture?

BACKGROUND

While machine learning (ML) algorithms may predict cardiovascular outcomes more accurately than statistical models, their result is usually not representable by a transparent formula. Hence, it is often unclear how specific values of predictors lead to the predictions. We aimed to demonstrate with graphical tools how predictor-risk relations in cardiovascular risk prediction models fitted by ML algorithms and by statistical approaches may differ, and how sample size affects the stability of the estimated relations.

METHODS

We reanalyzed data from a large registry of 1.5 million participants in a national health screening program. Three data analysts developed analytical strategies to predict cardiovascular events within 1 year from health screening. This was done for the full data set and with gradually reduced sample sizes, and each data analyst followed their favorite modeling approach. Predictor-risk relations were visualized by partial dependence and individual conditional expectation plots.

RESULTS

When comparing the modeling algorithms, we found some similarities between these visualizations but also occasional divergence. The smaller the sample size, the more the predictor-risk relation depended on the modeling algorithm used, and also sampling variability played an increased role. Predictive performance was similar if the models were derived on the full data set, whereas smaller sample sizes favored simpler models.

CONCLUSION

Predictor-risk relations from ML models may differ from those obtained by statistical models, even with large sample sizes. Hence, predictors may assume different roles in risk prediction models. As long as sample size is sufficient, predictive accuracy is not largely affected by the choice of algorithm.

Prediction of coronary heart disease in rural Chinese adults: a cross sectional study

Background

Coronary heart disease (CHD) is a common cardiovascular disease with high morbidity and mortality in China. The CHD risk prediction model has a great value in early prevention and diagnosis.

Methods

In this study, CHD risk prediction models among rural residents in Xinxiang County were constructed using Random Forest (RF), Support Vector Machine (SVM), and the least absolute shrinkage and selection operator (LASSO) regression algorithms with identified 16 influencing factors.

Results

Results demonstrated that the CHD model using the RF classifier performed best both on the training set and test set, with the highest area under the curve (AUC = 1 and 0.9711), accuracy (one and 0.9389), sensitivity (one and 0.8725), specificity (one and 0.9771), precision (one and 0.9563), F1-score (one and 0.9125), and Matthews correlation coefficient (MCC = one and 0.8678), followed by the SVM (AUC = 0.9860 and 0.9589) and the LASSO classifier (AUC = 0.9733 and 0.9587). Besides, the RF model also had an increase in the net reclassification index (NRI) and integrated discrimination improvement (IDI) values, and achieved a greater net benefit in the decision curve analysis (DCA) compared with the SVM and LASSO models.

Conclusion

The CHD risk prediction model constructed by the RF algorithm in this study is conducive to the early diagnosis of CHD in rural residents of Xinxiang County, Henan Province.

Selection of variables for multivariable models: Opportunities and limitations in quantifying model stability by resampling

Statistical models are often fitted to obtain a concise description of the association of an outcome variable with some covariates. Even if background knowledge is available to guide preselection of covariates, stepwise variable selection is commonly applied to remove irrelevant ones. This practice may introduce additional variability and selection is rarely certain. However, these issues are often ignored and model stability is not questioned. Several resampling-based measures were proposed to describe model stability, including variable inclusion frequencies (VIFs), model selection frequencies, relative conditional bias (RCB), and root mean squared difference ratio (RMSDR). The latter two were recently proposed to assess bias and variance inflation induced by variable selection. Here, we study the consistency and accuracy of resampling estimates of these measures and the optimal choice of the resampling technique. In particular, we compare subsampling and bootstrapping for assessing stability of linear, logistic, and Cox models obtained by backward elimination in a simulation study. Moreover, we exemplify the estimation and interpretation of all suggested measures in a study on cardiovascular risk. The VIF and the model selection frequency are only consistently estimated in the subsampling approach. By contrast, the bootstrap is advantageous in terms of bias and precision for estimating the RCB as well as the RMSDR. Though, unbiased estimation of the latter quantity requires independence of covariates, which is rarely encountered in practice. Our study stresses the importance of addressing model stability after variable selection and shows how to cope with it.