Quantifying and Interpreting the Prediction Accuracy of Models for the Time of a Cardiovascular Event-Moving Beyond C Statistic: A Review

Development and Validation of a Machine Learning-Based Prognostic Model for IgA Nephropathy with Chronic Kidney Disease Stage 3 or 4

Introduction

Immunoglobulin A nephropathy (IgAN) patients with lower estimated glomerular filtration rate (eGFR) and higher proteinuria are at a higher risk for end-stage kidney disease (ESKD) and their prognosis is still unclear. We aim to develop and validate prognostic models in IgAN patients with chronic kidney disease (CKD) stage 3 or 4 and proteinuria ≥1.0 g/d.

Methods

Patients who came from Xijing Hospital, spanning December 2008 to January 2020 were divided into training and test cohorts randomly, with a ratio of 7:3, achieving ESKD and death as study endpoints. Created prediction models for IgAN patients based on 66 clinical and pathological characteristics using the random survival forests (RSF), survival support vector machine (SSVM), eXtreme Gradient Boosting (XGboost), and Cox regression models. The concordance index (C-index), integrated Brier scores (IBS), net reclassification index (NRI), and integrated discrimination improvement (IDI) were used to evaluate discrimination, calibration, and risk classification, respectively.

Results

A total of 263 patients were enrolled. The median follow-up time was 57.3 months, with 124 (47.1%) patients experiencing combined events. Age, blood urea nitrogen, serum uric acid, serum potassium, glomeruli sclerosis ratio, hemoglobin, and tubular atrophy/interstitial fibrosis were identified as risk factors. The RSF model predicted the prognosis with a C-index of 0.871 (0.842, 0.900) in training cohort and 0.810 (0.732, 0.888) in test cohort, which was higher than the models built by SSVM model (0.794 [0.753, 0.835] and 0.805 [0.731, 0.879], respectively), XGboost model (0.840 [0.797, 0.883] and 0.799 [0.723, 0.875], respectively) and Cox regression (0.776 [0.727, 0.825] and 0.793 [0.713, 0.873], respectively). NRI and IDI showed that the RSF model exhibited superior performance than the Cox model.

Conclusion

Our model introduced seven risk factors that may be useful in predicting the progression of IgAN patients with CKD stage 3 or 4 and proteinuria ≥1.0 g/d. The RSF model is applicable for identifying the progression of IgAN and has outperformed than SSVM, XGboost, and Cox models.

Deep learning models for predicting the survival of patients with medulloblastoma based on a surveillance, epidemiology, and end results analysis

Medulloblastoma is a malignant neuroepithelial tumor of the central nervous system. Accurate prediction of prognosis is essential for therapeutic decisions in medulloblastoma patients. We analyzed data from 2,322 medulloblastoma patients using the SEER database and randomly divided the dataset into training and testing datasets in a 7:3 ratio. We chose three models to build, one based on neural networks (DeepSurv), one based on ensemble learning that Random Survival Forest (RSF), and a typical Cox Proportional-hazards (CoxPH) model. The DeepSurv model outperformed the RSF and classic CoxPH models with C-indexes of 0.751 and 0.763 for the training and test datasets. Additionally, the DeepSurv model showed better accuracy in predicting 1-, 3-, and 5-year survival rates (AUC: 0.767-0.793). Therefore, our prediction model based on deep learning algorithms can more accurately predict the survival rate and survival period of medulloblastoma compared to other models.

A risk stratification model modified from the U.S. guideline could be applied in an Asian population with or without ASCVD: Validation study

BACKGROUND

This study aimed to evaluate the performance of a modified U.S. (MUS) model for risk prediction of cardiovascular (CV) events in Asian patients and compare it to European and Japanese models.

METHODS

The MUS model, based on the US ACC/AHA 2018 lipid treatment guideline, was employed to stratify patients under primary or secondary prevention. Two multi-center prospective observational registry cohorts, T-SPARCLE and T-PPARCLE, were used to validate the scoring system, and the primary outcome was the time to first occurrence/recurrence of major adverse cardiac events (MACEs). The MUS model's performance was compared to other models from Europe and Japan.

RESULTS

A total of 10,733 patients with the mean age of 64.2 (SD: 11.9) and 36.5% female were followed up for a median of 5.4 years. The MUS model was validated, with an AUC score of 0.73 (95% CI 0.68-0.78). The European and Japanese models had AUC scores ranging from 0.6 to 0.7. The MUS model categorized patients into four distinct CV risk groups, with hazard ratios (HRs) as follows: very high- vs. high-risk group (HR = 1.91, 95% CI 1.53-2.39), high- vs. moderate-risk group (HR = 2.08, 95% CI 1.60-2.69), and moderate- vs. low-risk group (HR = 3.14, 95% CI 1.63-6.03). After adjusting for the MUS model, a history of atherosclerotic vascular disease (ASCVD) was not a significant predictor of adverse cardiovascular outcomes within each risk group.

CONCLUSION

The MUS model is an effective tool for risk stratification in Asian patients with and without ASCVD, accurately predicting MACEs and performing comparably or better than other established risk models. Our findings suggest that patient management should focus on background risk factors instead of solely on primary or secondary prevention.

Predicting the risk of 1-year mortality among patients hospitalized for acute heart failure in China

BACKGROUND

We aimed to develop and validate a model to predict 1-year mortality risk among patients hospitalized for acute heart failure (AHF), build a risk score and interpret its application in clinical decision making.

METHODS

By using data from China Patient-Centred Evaluative Assessment of Cardiac Events Prospective Heart Failure Study, which prospectively enrolled patients hospitalized for AHF in 52 hospitals across 20 provinces, we used multivariate Cox proportional hazard model to develop and validate a model to predict 1-year mortality.

RESULTS

There were 4,875 patients included in the study, 857 (17.58%) of them died within 1-year following discharge of index hospitalization. A total of 13 predictors were selected to establish the prediction model, including age, medical history of chronic obstructive pulmonary disease and hypertension, systolic blood pressure, Kansas City Cardiomyopathy Questionnaire-12 score, angiotensin converting enzyme inhibitor or angiotensin receptor blocker at discharge, discharge symptom, N-terminal pro-brain natriuretic peptide, high-sensitivity troponin T, serum creatine, albumin, blood urea nitrogen, and highly sensitive C-reactive protein. The model showed a high performance on discrimination (C-index was 0.759 [95% confidence interval: 0.739, 0.778] in development cohort and 0.761 [95% confidence interval: 0.731, 0.791] in validation cohort), accuracy, calibration, and outperformed than several existed risk scores. A point-based risk score was built to stratify low- (0-12), intermediate- (13-16), and high-risk group (≥17) among patients.

CONCLUSIONS

A prediction model using readily available predictors was developed and internal validated to predict 1-year mortality risk among patients hospitalized for AHF. It may serve as a useful tool for individual risk stratification and informing decision making to improve clinical care.

Normative Values of Echocardiographic Chamber Size and Function in Older Healthy Adults: The Multi-Ethnic Study of Atherosclerosis

BACKGROUND

Echocardiographic (2-dimensional echocardiography) thresholds indicating disease or impaired functional status compared with normal physiological aging in individuals aged ≥65 years are not clearly defined. In the present study, we sought to establish standard values for 2-dimensional echocardiography parameters related to chamber size and function in older adults without cardiopulmonary or cardiometabolic conditions.

METHODS

In this cross-sectional study of 3032 individuals who underwent 2-dimensional echocardiography at exam 6 in the MESA (Multi-Ethnic Study of Atherosclerosis), 608 participants fulfilled our inclusion criteria of healthy aging, with normative values defined as the mean ± 1.96 standard deviation and compared across sex and race and ethnicity. Functional status measures included NT-proBNP (N-terminal pro-B-type natriuretic peptide), 6-minute walk distance, and Kansas City Cardiomyopathy Questionnaire. Prognostic performance using MESA cutoffs was compared with established guideline cutoffs using time-to-event analysis.

RESULTS

The normative aging cohort (69.5±7.0 years, 46.2% male, 47.5% White) had lower NT-proBNP, higher 6-minute walk distance, and higher (better) Kansas City Cardiomyopathy Questionnaire summary values. Women had significantly smaller chamber sizes and better biventricular systolic function. White participants had the largest chamber dimensions, whereas Chinese participants had the smallest, even after adjustment for body size. Current guidelines identified 81.6% of healthy older adults in MESA as having cardiac abnormalities.

CONCLUSIONS

Among a large, diverse group of healthy older adults, we found significant differences in cardiac structure and function by sex and race/ethnicity, which may signal sex-specific cardiac remodeling with advancing age. It is crucial for existing guidelines to consider the observed and clinically significant differences in cardiac structure and function associated with healthy aging. Our study highlights that existing guidelines, which grade abnormalities in echocardiographic cardiac chamber size and function based on younger individuals, may not adequately address the anticipated changes associated with normal aging.

Impact of age on the predictive value of NT-proBNP in patients with diabetes mellitus stabilised after an acute coronary syndrome

AIMS

To assess the impact of age on the prognostic value of NT-proBNP concentration in patients with type-2 diabetes mellitus (T2DM) stabilised after an Acute Coronary Syndrome (ACS).

METHODS

The AleCardio study compared aleglitazar with placebo in 7226 patients with T2DM and recent ACS. Patients with heart failure were excluded. Median follow-up was 104 weeks. Baseline NT-proBNP plasma concentration was measured centrally. Multivariable Cox regression was used to determine the mortality predictive information provided by NT-proBNP across age groups.

RESULTS

Median age was 61y (IQR 54, 67). NT-proBNP concentration increased by quartile (Q) of age (median 264, 318, 391, and 588 pg/ml). Compared to Q1, patients in Q4 of NT-proBNP had higher (p < 0.001) adjusted HR for all-cause (aHR 6.9; 95 % CI 4.0-12) and cardiovascular (11; 5.4-23) death. Within each age Q, baseline NT-proBNP in patients who died was 3 times higher than in survivors (all p < 0.001). When age and NT-proBNP levels were modeled as continuous variables, their interaction term was nonsignificant. The relative prognostic information provided by NT-proBNP (percent of total X2) increased from 38 % in age Q1 to 75 % in age Q4 for mortality, and from 50 % to 88 % for CV death.

CONCLUSIONS

Among patients with T2DM stabilised after an ACS, NT-proBNP level predicts death irrespective of age.

Normative Values of Echocardiographic Chamber Size and Function in Older Healthy Adults: The Multi-Ethnic Study of Atherosclerosis

Background

Echocardiographic (2DE) thresholds indicating disease or impaired functional status compared to normal physiologic aging in individuals ≥ 65 years are not clearly defined. In the present study, we sought to establish standard values for 2DE parameters related to chamber size and function in older adults without cardiopulmonary or cardiometabolic conditions.

Methods

In this cross-sectional study of 3032 individuals who underwent 2DE at Exam 6 in the Multi-Ethnic Study of Atherosclerosis (MESA), 608 participants fulfilled our inclusion criteria, with normative values defined as the mean value ± 1.96 standard deviations and compared across sex and race/ethnicity. Functional status measures included NT-proBNP, 6-minute walk distance [6MWD], and Kansas City Cardiomyopathy Questionnaire [KCCQ]. Prognostic performance using MESA cutoffs was compared to established guideline cutoffs using time-to-event analysis.

Results

Participants meeting our inclusion criteria (69.5 ± 7.0 years, 46.2% male, 47.5% White) had lower NT-proBNP, higher 6MWD, and higher (better) KCCQ summary values. Women had significantly smaller chamber sizes and better biventricular systolic function. White participants had the largest chamber dimensions, while Chinese participants had the smallest, even after adjustment for body size. Current guidelines identified 81.6% of healthy older adults in MESA as having cardiac abnormalities.

Conclusions

Among a large, diverse group of healthy older adults, we found significant differences in cardiac structure and function across sexes and races/ethnicities, which may signal sex-specific cardiac remodeling with advancing age. It is crucial for existing guidelines to consider the observed and clinically significant differences in cardiac structure and function associated with healthy aging. Our study highlights that existing guidelines, which grade abnormalities in echocardiographic cardiac chamber size and function based on younger individuals, may not adequately address the anticipated changes associated with normal aging.