Prognostic evaluation by clinical exercise test scores in patients treated with primary percutaneous coronary intervention or fibrinolysis for acute myocardial infarction (a Danish Trial in Acute Myocardial Infarction-2 Sub-Study)

External validation of new risk prediction models is infrequent and reveals worse prognostic discrimination

OBJECTIVES

To evaluate how often newly developed risk prediction models undergo external validation and how well they perform in such validations.

STUDY DESIGN AND SETTING

We reviewed derivation studies of newly proposed risk models and their subsequent external validations. Study characteristics, outcome(s), and models' discriminatory performance [area under the curve, (AUC)] in derivation and validation studies were extracted. We estimated the probability of having a validation, change in discriminatory performance with more stringent external validation by overlapping or different authors compared to the derivation estimates.

RESULTS

We evaluated 127 new prediction models. Of those, for 32 models (25%), at least an external validation study was identified; in 22 models (17%), the validation had been done by entirely different authors. The probability of having an external validation by different authors within 5 years was 16%. AUC estimates significantly decreased during external validation vs. the derivation study [median AUC change: -0.05 (P < 0.001) overall; -0.04 (P = 0.009) for validation by overlapping authors; -0.05 (P < 0.001) for validation by different authors]. On external validation, AUC decreased by at least 0.03 in 19 models and never increased by at least 0.03 (P < 0.001).

CONCLUSION

External independent validation of predictive models in different studies is uncommon. Predictive performance may worsen substantially on external validation.

Neonatal intensive care unit: predictive models for length of stay

OBJECTIVE

Hospital length of stay (LOS) is important to administrators and families of neonates admitted to the neonatal intensive care unit (NICU). A prediction model for NICU LOS was developed using predictors birth weight, gestational age and two severity of illness tools, the score for neonatal acute physiology, perinatal extension (SNAPPE) and the morbidity assessment index for newborns (MAIN).

STUDY DESIGN

Consecutive admissions (n=293) to a New England regional level III NICU were retrospectively collected. Multiple predictive models were compared for complexity and goodness-of-fit, coefficient of determination (R (2)) and predictive error. The optimal model was validated prospectively with consecutive admissions (n=615). Observed and expected LOS was compared.

RESULT

The MAIN models had best Akaike's information criterion, highest R (2) (0.786) and lowest predictive error. The best SNAPPE model underestimated LOS, with substantial variability, yet was fairly well calibrated by birthweight category. LOS was longer in the prospective cohort than the retrospective cohort, without differences in birth weight, gestational age, MAIN or SNAPPE.

CONCLUSION

LOS prediction is improved by accounting for severity of illness in the first week of life, beyond factors known at birth. Prospective validation of both MAIN and SNAPPE models is warranted.

Relationship between changes in heart rate recovery after cardiac rehabilitation on cardiovascular mortality in patients with myocardial infarction

BACKGROUND

Heart rate recovery (HRR) at predischarge exercise stress test predicts all-cause mortality in patients with myocardial infarction (MI), but the relationship between improvement in HRR with exercise training and clinical outcomes remains unclear.

OBJECTIVE

The purpose of this study was to evaluate the effect of change in HRR after exercise training on clinical outcomes in MI patients.

METHODS

The study included 386 consecutive patients with recent MI who were enrolled into our cardiac rehabilitation program. All patients underwent symptom-limited treadmill testing at baseline and after exercise training, and were prospectively followed-up in the outpatient clinic.

RESULTS

Treadmill testing revealed significant improvement in HRR after 8 weeks of exercise training (17.5 +/- 10.0 bpm to 19.0 +/- 12.3 bpm, P = .011). After follow-up of 79 +/- 41 months, 40 (10.4%) patients died of cardiac events. Multivariate Cox regression analysis revealed that diabetes (hazard ratio [HR] 2.28, 95% confidence interval [CI] 1.01-5.19, P = .049), statin use (HR 0.36, 95% CI 0.16-0.80, P = .012), baseline resting heart rate > or =65 bpm (HR 5.37, 95% CI 1.33-21.61, P = .018), post-training HRR <12 bpm (HR 2.49, 95% CI 1.10-5.63, P = .028), left ventricular ejection fraction < or =30% (HR 4.70, 95% CI 1.34-16.46, P = .016), and exercise capacity < or =4 metabolic equivalents (HR 3.63, 95% CI 1.17-11.28, P = .026) were independent predictors of cardiac death. Patients who failed to improve HRR from <12 bpm to > or =12 bpm after exercise training had significantly higher mortality (HR 6.2, 95% CI 1.3-29.2, P = .022).

CONCLUSION

Exercise training improved HRR in patients with recent MI, and patients with HRR increased to > or =12 bpm had better cardiac survival.