Incremental Value of Repeated Risk Factor Measurements for Cardiovascular Disease Prediction in Middle-Aged Korean Adults: Results From the NHIS-HEALS (National Health Insurance System-National Health Screening Cohort)

Performance of Cardiovascular Risk Prediction Models in Korean Patients With New-Onset Rheumatoid Arthritis: National Cohort Study

BACKGROUND

This study aimed to compare the performance of established cardiovascular risk algorithms in Korean patients with new-onset rheumatoid arthritis.

METHODS AND RESULTS

This retrospective cohort study identified patients newly diagnosed with rheumatoid arthritis without a history of cardiovascular diseases between 2013 and 2019 using the National Health Insurance Service database. The cohort was followed up until 2020 for the development of the first major adverse cardiovascular event. General cardiovascular risk prediction algorithms, such as the systematic coronary risk evaluation model, the Korean risk prediction model for atherosclerotic cardiovascular diseases, the American College of Cardiology/American Heart Association pooled equations, and the Framingham Risk Score, were used. The discrimination and calibration of cardiovascular risk prediction models were evaluated. Hazard ratios were estimated using Cox proportional hazards regression. A total of 611 patients among 24 889 patients experienced a major adverse cardiovascular event during follow-up. The median 10-year atherosclerotic cardiovascular diseases risk score was significantly higher in patients with major adverse cardiovascular events than those without. The C-statistics of risk algorithms ranged between 0.72 and 0.74. Compared with the low-risk group, the actual risk of developing major adverse cardiovascular events increased significantly in the intermediate- and high-risk groups for all algorithms. However, the risk predictions calculated from all algorithms overestimated the observed cardiovascular risk in the middle to high deciles, and only the systematic coronary risk evaluation algorithm showed comparable observed and predicted event rates in the low-intermediate deciles with the highest sensitivity.

CONCLUSIONS

The systematic coronary risk evaluation model algorithm and the general risk prediction models discriminated patients with rheumatoid arthritis appropriately. However, overestimation should be considered when applying the cardiovascular risk prediction model in Korean patients.

Cholesterol and Hypertension Treatment Improve Coronary Risk Prediction but Not Time-Dependent Covariates or Competing Risks

Background and Aims

Cardiovascular (CV) risk functions are the recommended tool to identify high-risk individuals. However, their discrimination ability is not optimal. While the effect of biomarkers in CV risk prediction has been extensively studied, there are no data on CV risk functions including time-dependent covariates together with other variables. Our aim was to examine the effect of including time-dependent covariates, competing risks, and treatments in coronary risk prediction.

Methods

Participants from the REGICOR population cohorts (North-Eastern Spain) aged 35-74 years without previous history of cardiovascular disease were included (n = 8470). Coronary and stroke events and mortality due to other CV causes or to cancer were recorded during follow-up (median = 12.6 years). A multi-state Markov model was constructed to include competing risks and time-dependent classical risk factors and treatments (2 measurements). This model was compared to Cox models with basal measurement of classical risk factors, treatments, or competing risks. Models were cross-validated and compared for discrimination (area under ROC curve), calibration (Hosmer-Lemeshow test), and reclassification (categorical net reclassification index).

Results

Cancer mortality was the highest cumulative-incidence event. Adding cholesterol and hypertension treatment to classical risk factors improved discrimination of coronary events by 2% and reclassification by 7-9%. The inclusion of competing risks and/or 2 measurements of risk factors provided similar coronary event prediction, compared to a single measurement of risk factors.

Conclusion

Coronary risk prediction improves when cholesterol and hypertension treatment are included in risk functions. Coronary risk prediction does not improve with 2 measurements of covariates or inclusion of competing risks.

Longitudinal trajectories of blood lipid levels in an ageing population sample of Russian Western-Siberian urban population

This study investigated 12-year blood lipid trajectories and whether these trajectories are modified by smoking and lipid lowering treatment in older Russians. To do so, we analysed data on 9,218 Russian West-Siberian Caucasians aged 45-69 years at baseline participating in the international HAPIEE cohort study. Mixed-effect multilevel models were used to estimate individual level lipid trajectories across the baseline and two follow-up examinations (16,445 separate measurements over 12 years). In all age groups, we observed a reduction in serum total cholesterol (TC), LDL-C and non-HDL-C over time even after adjusting for sex, statin treatment, hypertension, diabetes, social factors and mortality (P<0.01). In contrast, serum triglyceride (TG) values increased over time in younger age groups, reached a plateau and decreased in older age groups (> 60 years at baseline). In smokers, TC, LDL-C, non-HDL-C and TG decreased less markedly than in non-smokers, while HDL-C decreased more rapidly while the LDL-C/HDL-C ratio increased. In subjects treated with lipid-lowering drugs, TC, LDL-C and non-HDL-C decreased more markedly and HDL-C less markedly than in untreated subjects while TG and LDL-C/HDL-C remained stable or increased in treatment naïve subjects. We conclude, that in this ageing population we observed marked changes in blood lipids over a 12 year follow up, with decreasing trajectories of TC, LDL-C and non-HDL-C and mixed trajectories of TG. The findings suggest that monitoring of age-related trajectories in blood lipids may improve prediction of CVD risk beyond single measurements.

Utility of single versus sequential measurements of risk factors for prediction of stroke in Chinese adults

Absolute risks of stroke are typically estimated using measurements of cardiovascular disease risk factors recorded at a single visit. However, the comparative utility of single versus sequential risk factor measurements for stroke prediction is unclear. Risk factors were recorded on three separate visits on 13,753 individuals in the prospective China Kadoorie Biobank. All participants were stroke-free at baseline (2004-2008), first resurvey (2008), and second resurvey (2013-2014), and were followed-up for incident cases of first stroke in the 3 years following the second resurvey. To reflect the models currently used in clinical practice, sex-specific Cox models were developed to estimate 3-year risks of stroke using single measurements recorded at second resurvey and were retrospectively applied to risk factor data from previous visits. Temporal trends in the Cox-generated risk estimates from 2004 to 2014 were analyzed using linear mixed effects models. To assess the value of more flexible machine learning approaches and the incorporation of longitudinal data, we developed gradient boosted tree (GBT) models for 3-year prediction of stroke using both single measurements and sequential measurements of risk factor inputs. Overall, Cox-generated estimates for 3-year stroke risk increased by 0.3% per annum in men and 0.2% per annum in women, but varied substantially between individuals. The risk estimates at second resurvey were highly correlated with the annual increase of risk for each individual (men: r = 0.91, women: r = 0.89), and performance of the longitudinal GBT models was comparable with both Cox and GBT models that considered measurements from only a single visit (AUCs: 0.779-0.811 in men, 0.724-0.756 in women). These results provide support for current clinical guidelines, which recommend using risk factor measurements recorded at a single visit for stroke prediction.

Development and Validation of a Deep Learning Based Diabetes Prediction System Using a Nationwide Population-Based Cohort

BACKGROUND

Previously developed prediction models for type 2 diabetes mellitus (T2DM) have limited performance. We developed a deep learning (DL) based model using a cohort representative of the Korean population.

METHODS

This study was conducted on the basis of the National Health Insurance Service-Health Screening (NHIS-HEALS) cohort of Korea. Overall, 335,302 subjects without T2DM at baseline were included. We developed the model based on 80% of the subjects, and verified the power in the remainder. Predictive models for T2DM were constructed using the recurrent neural network long short-term memory (RNN-LSTM) network and the Cox longitudinal summary model. The performance of both models over a 10-year period was compared using a time dependent area under the curve.

RESULTS

During a mean follow-up of 10.4±1.7 years, the mean frequency of periodic health check-ups was 2.9±1.0 per subject. During the observation period, T2DM was newly observed in 8.7% of the subjects. The annual performance of the model created using the RNN-LSTM network was superior to that of the Cox model, and the risk factors for T2DM, derived using the two models were similar; however, certain results differed.

CONCLUSION

The DL-based T2DM prediction model, constructed using a cohort representative of the population, performs better than the conventional model. After pilot tests, this model will be provided to all Korean national health screening recipients in the future.

Impact of obesity, fasting plasma glucose level, blood pressure, and renal function on the severity of COVID-19: A matter of sexual dimorphism?

AIMS

This study aimed to assess whether body mass index (BMI), fasting plasma glucose (FPG) levels, blood pressure (BP), and kidney function were associated with the risk of severe disease or death in patients with COVID-19.

METHODS

Data on candidate risk factors were extracted from patients' last checkup records. Propensity score-matched cohorts were constructed, and logistic regression models were used to adjust for age, sex, and comorbidities. The primary outcome was death or severe COVID-19, defined as requiring supplementary oxygen or higher ventilatory support.

RESULTS

Among 7,649 patients with confirmed COVID-19, 2,231 (29.2%) received checkups and severe COVID-19 occurred in 307 patients (13.8%). A BMI of 25.0-29.9 was associated with the outcome among women (aOR, 2.29; 95% CI, 1.41-3.73) and patients aged 50-69 years (aOR, 1.64; 95% CI, 1.06-2.54). An FPG ≥ 126 mg/dL was associated with poor outcomes in women (aOR, 2.06; 95% CI, 1.13-3.77) but not in men. Similarly, estimated glomerular filtration rate (eGFR) < 60 ml/min/1.73 m2 was a risk factor in women (aOR, 3.46; 95% CI, 1.71-7.01) and patients aged < 70 years.

CONCLUSIONS

The effects of BMI, FPG, and eGFR on outcomes associated with COVID-19 were prominent in women but not in men.

Harnessing repeated measurements of predictor variables for clinical risk prediction: a review of existing methods

BACKGROUND

Clinical prediction models (CPMs) predict the risk of health outcomes for individual patients. The majority of existing CPMs only harness cross-sectional patient information. Incorporating repeated measurements, such as those stored in electronic health records, into CPMs may provide an opportunity to enhance their performance. However, the number and complexity of methodological approaches available could make it difficult for researchers to explore this opportunity. Our objective was to review the literature and summarise existing approaches for harnessing repeated measurements of predictor variables in CPMs, primarily to make this field more accessible for applied researchers.

METHODS

MEDLINE, Embase and Web of Science were searched for articles reporting the development of a multivariable CPM for individual-level prediction of future binary or time-to-event outcomes and modelling repeated measurements of at least one predictor. Information was extracted on the following: the methodology used, its specific aim, reported advantages and limitations, and software available to apply the method.

RESULTS

The search revealed 217 relevant articles. Seven methodological frameworks were identified: time-dependent covariate modelling, generalised estimating equations, landmark analysis, two-stage modelling, joint-modelling, trajectory classification and machine learning. Each of these frameworks satisfies at least one of three aims: to better represent the predictor-outcome relationship over time, to infer a covariate value at a pre-specified time and to account for the effect of covariate change.

CONCLUSIONS

The applicability of identified methods depends on the motivation for including longitudinal information and the method's compatibility with the clinical context and available patient data, for both model development and risk estimation in practice.

Development and verification of prediction models for preventing cardiovascular diseases

Objectives

Cardiovascular disease (CVD) is one of the major causes of death worldwide. For improved accuracy of CVD prediction, risk classification was performed using national time-series health examination data. The data offers an opportunity to access deep learning (RNN-LSTM), which is widely known as an outstanding algorithm for analyzing time-series datasets. The objective of this study was to show the improved accuracy of deep learning by comparing the performance of a Cox hazard regression and RNN-LSTM based on survival analysis.

Methods and findings

We selected 361,239 subjects (age 40 to 79 years) with more than two health examination records from 2002–2006 using the National Health Insurance System-National Health Screening Cohort (NHIS-HEALS). The average number of health screenings (from 2002–2013) used in the analysis was 2.9 ± 1.0. Two CVD prediction models were developed from the NHIS-HEALS data: a Cox hazard regression model and a deep learning model. In an internal validation of the NHIS-HEALS dataset, the Cox regression model showed a highest time-dependent area under the curve (AUC) of 0.79 (95% CI 0.70 to 0.87) for in females and 0.75 (95% CI 0.70 to 0.80) in males at 2 years. The deep learning model showed a highest time-dependent AUC of 0.94 (95% CI 0.91 to 0.97) for in females and 0.96 (95% CI 0.95 to 0.97) in males at 2 years. Layer-wise Relevance Propagation (LRP) revealed that age was the variable that had the greatest effect on CVD, followed by systolic blood pressure (SBP) and diastolic blood pressure (DBP), in that order.

Conclusion

The performance of the deep learning model for predicting CVD occurrences was better than that of the Cox regression model. In addition, it was confirmed that the known risk factors shown to be important by previous clinical studies were extracted from the study results using LRP.

Cox Proportional Hazard Regression Versus a Deep Learning Algorithm in the Prediction of Dementia: An Analysis Based on Periodic Health Examination

Background

With the increase in the world’s aging population, there is a growing need to prevent and predict dementia among the general population. The availability of national time-series health examination data in South Korea provides an opportunity to use deep learning algorithm, an artificial intelligence technology, to expedite the analysis of mass and sequential data.

Objective

This study aimed to compare the discriminative accuracy between a time-series deep learning algorithm and conventional statistical methods to predict all-cause dementia and Alzheimer dementia using periodic health examination data.

Methods

Diagnostic codes in medical claims data from a South Korean national health examination cohort were used to identify individuals who developed dementia or Alzheimer dementia over a 10-year period. As a result, 479,845 and 465,081 individuals, who were aged 40 to 79 years and without all-cause dementia and Alzheimer dementia, respectively, were identified at baseline. The performance of the following 3 models was compared with predictions of which individuals would develop either type of dementia: Cox proportional hazards model using only baseline data (HR-B), Cox proportional hazards model using repeated measurements (HR-R), and deep learning model using repeated measurements (DL-R).

Results

The discrimination indices (95% CI) for the HR-B, HR-R, and DL-R models to predict all-cause dementia were 0.84 (0.83-0.85), 0.87 (0.86-0.88), and 0.90 (0.90-0.90), respectively, and those to predict Alzheimer dementia were 0.87 (0.86-0.88), 0.90 (0.88-0.91), and 0.91 (0.91-0.91), respectively. The DL-R model showed the best performance, followed by the HR-R model, in predicting both types of dementia. The DL-R model was superior to the HR-R model in all validation groups tested.

Conclusions

A deep learning algorithm using time-series data can be an accurate and cost-effective method to predict dementia. A combination of deep learning and proportional hazards models might help to enhance prevention strategies for dementia.

Development and External Validation of a Deep Learning Algorithm for Prognostication of Cardiovascular Outcomes

Background and Objectives

We aim to explore the additional discriminative accuracy of a deep learning (DL) algorithm using repeated-measures data for identifying people at high risk for cardiovascular disease (CVD), compared to Cox hazard regression.

Methods

Two CVD prediction models were developed from National Health Insurance Service-Health Screening Cohort (NHIS-HEALS): a Cox regression model and a DL model. Performance of each model was assessed in the internal and 2 external validation cohorts in Koreans (National Health Insurance Service-National Sample Cohort; NHIS-NSC) and in Europeans (Rotterdam Study). A total of 412,030 adults in the NHIS-HEALS; 178,875 adults in the NHIS-NSC; and the 4,296 adults in Rotterdam Study were included.

Results

Mean ages was 52 years (46% women) and there were 25,777 events (6.3%) in NHIS-HEALS during the follow-up. In internal validation, the DL approach demonstrated a C-statistic of 0.896 (95% confidence interval, 0.886–0.907) in men and 0.921 (0.908–0.934) in women and improved reclassification compared with Cox regression (net reclassification index [NRI], 24.8% in men, 29.0% in women). In external validation with NHIS-NSC, DL demonstrated a C-statistic of 0.868 (0.860–0.876) in men and 0.889 (0.876–0.898) in women, and improved reclassification compared with Cox regression (NRI, 24.9% in men, 26.2% in women). In external validation applied to the Rotterdam Study, DL demonstrated a C-statistic of 0.860 (0.824–0.897) in men and 0.867 (0.830–0.903) in women, and improved reclassification compared with Cox regression (NRI, 36.9% in men, 31.8% in women).

Conclusions

A DL algorithm exhibited greater discriminative accuracy than Cox model approaches.

Trial Registration

ClinicalTrials.gov Identifier: NCT02931500