Potential demographic and baselines variables for risk stratification of high-risk post-myocardial infarction patients in the era of implantable cardioverter-defibrillator--a prognostic indicator

Generalizability of Cardiovascular Disease Clinical Prediction Models: 158 Independent External Validations of 104 Unique Models

Background:

While clinical prediction models (CPMs) are used increasingly commonly to guide patient care, the performance and clinical utility of these CPMs in new patient cohorts is poorly understood.

Methods:

We performed 158 external validations of 104 unique CPMs across 3 domains of cardiovascular disease (primary prevention, acute coronary syndrome, and heart failure). Validations were performed in publicly available clinical trial cohorts and model performance was assessed using measures of discrimination, calibration, and net benefit. To explore potential reasons for poor model performance, CPM-clinical trial cohort pairs were stratified based on relatedness, a domain-specific set of characteristics to qualitatively grade the similarity of derivation and validation patient populations. We also examined the model-based C-statistic to assess whether changes in discrimination were because of differences in case-mix between the derivation and validation samples. The impact of model updating on model performance was also assessed.

Results:

Discrimination decreased significantly between model derivation (0.76 [interquartile range 0.73–0.78]) and validation (0.64 [interquartile range 0.60–0.67], P<0.001), but approximately half of this decrease was because of narrower case-mix in the validation samples. CPMs had better discrimination when tested in related compared with distantly related trial cohorts. Calibration slope was also significantly higher in related trial cohorts (0.77 [interquartile range, 0.59–0.90]) than distantly related cohorts (0.59 [interquartile range 0.43–0.73], P=0.001). When considering the full range of possible decision thresholds between half and twice the outcome incidence, 91% of models had a risk of harm (net benefit below default strategy) at some threshold; this risk could be reduced substantially via updating model intercept, calibration slope, or complete re-estimation.

Conclusions:

There are significant decreases in model performance when applying cardiovascular disease CPMs to new patient populations, resulting in substantial risk of harm. Model updating can mitigate these risks. Care should be taken when using CPMs to guide clinical decision-making.

Appropriate secondary prevention and clinical outcomes after acute myocardial infarction according to atherothrombotic risk stratification: The FAST-MI 2010 registry

BACKGROUND

Full secondary prevention medication regimen is often under-prescribed after acute myocardial infarction.

DESIGN

The purpose of this study was to analyse the relationship between prescription of appropriate secondary prevention treatment at discharge and long-term clinical outcomes according to risk level defined by the Thrombolysis In Myocardial Infarction (TIMI) Risk Score for Secondary Prevention (TRS-2P) after acute myocardial infarction.

METHODS

We used data from the 2010 French Registry of Acute ST-Elevation or non-ST-elevation Myocardial Infarction (FAST-MI) registry, including 4169 consecutive acute myocardial infarction patients admitted to cardiac intensive care units in France. Level of risk was stratified in three groups using the TRS-2P score: group 1 (low-risk; TRS-2P=0/1); group 2 (intermediate-risk; TRS-2P=2); and group 3 (high-risk; TRS-2P≥3). Appropriate secondary prevention treatment was defined according to the latest guidelines (dual antiplatelet therapy and moderate/high dose statins for all; new-P2Y12 inhibitors, angiotensin-converting-enzyme inhibitor/angiotensin-receptor-blockers and beta-blockers as indicated).

RESULTS

Prevalence of groups 1, 2 and 3 was 46%, 25% and 29% respectively. Appropriate secondary prevention treatment at discharge was used in 39.5%, 37% and 28% of each group, respectively. After multivariate adjustment, evidence-based treatments at discharge were associated with lower rates of major adverse cardiovascular events (death, re-myocardial infarction or stroke) at five years especially in high-risk patients: hazard ratio = 0.82 (95% confidence interval: 0.59-1.12, p = 0.21) in group 1, 0.74 (0.54-1.01; p = 0.06) in group 2, and 0.64 (0.52-0.79, p < 0.001) in group 3.

CONCLUSIONS

Use of appropriate secondary prevention treatment at discharge was inversely correlated with patient risk. The increased hazard related to lack of prescription of recommended medications was much larger in high-risk patients. Specific efforts should be directed at better prescription of recommended treatment, particularly in high-risk patients.

Long-term mortality risk in patients with an implantable cardioverter-defibrillator: Influence of heart rate and QRS duration

BACKGROUND

A paucity of studies has investigated the role of autonomic cardiac regulation as well as cardiac conduction in relation to prognosis in implantable cardioverter-defibrillator (ICD) patients. Therefore, we examined the association of heart rate and QRS duration with long-term mortality risk in first-time ICD patients, adjusting also for measures of emotional distress.

METHODS

Resting heart rate and QRS duration were assessed prior to ICD implantation in 448 patients. Primary study endpoint was all-cause mortality (up to 6.0 year follow-up, median follow-up of 5.6 years (IQR: 1.9)). The impact of heart rate and QRS duration on time to all-cause mortality was separately assessed with Cox proportional hazard regression analysis, adjusting for clinical factors and symptoms of depression and anxiety.

RESULTS

Mean (SD) heart rate was 68.0 ± 13.3 bpm and mean QRS duration was 130.9 ± 36.9 ms. Heart rate of ≥80 bpm was associated with increased risk of mortality (HR=1.86; 95% CI=1.15-3.00; p=.011) in unadjusted analysis. In adjusted analyses, this relationship remained significant both with depression (HR=1.86, 95% CI=1.12-3.09; p=.017) and anxiety (HR=1.82, 95% CI=1.10-3.03; p=.021) and clinical measures as covariates. QRS duration of ≥120 ms was associated with impaired prognosis in unadjusted analysis (HR=2.00, 95% CI=1.27-3.14; p=.003), but was reduced to non-significance in adjusted analysis when medical comorbidities were included (HR=1.15, 95% CI=0.70-1.89; p=.60).

CONCLUSIONS

This study shows that increased heart rate is associated with impaired prognosis. Since heart rate is a relatively easy measurable parameter of autonomic functioning, heart rate should be included as a measure for risk stratification in daily clinical practice.

Developing and validating risk prediction models in an individual participant data meta-analysis

Background

Risk prediction models estimate the risk of developing future outcomes for individuals based on one or more underlying characteristics (predictors). We review how researchers develop and validate risk prediction models within an individual participant data (IPD) meta-analysis, in order to assess the feasibility and conduct of the approach.

Methods

A qualitative review of the aims, methodology, and reporting in 15 articles that developed a risk prediction model using IPD from multiple studies.

Results

The IPD approach offers many opportunities but methodological challenges exist, including: unavailability of requested IPD, missing patient data and predictors, and between-study heterogeneity in methods of measurement, outcome definitions and predictor effects. Most articles develop their model using IPD from all available studies and perform only an internal validation (on the same set of data). Ten of the 15 articles did not allow for any study differences in baseline risk (intercepts), potentially limiting their model’s applicability and performance in some populations. Only two articles used external validation (on different data), including a novel method which develops the model on all but one of the IPD studies, tests performance in the excluded study, and repeats by rotating the omitted study.

Conclusions

An IPD meta-analysis offers unique opportunities for risk prediction research. Researchers can make more of this by allowing separate model intercept terms for each study (population) to improve generalisability, and by using ‘internal-external cross-validation’ to simultaneously develop and validate their model. Methodological challenges can be reduced by prospectively planned collaborations that share IPD for risk prediction.

Individual participant data meta-analysis of prognostic factor studies: state of the art?

Background

Prognostic factors are associated with the risk of a subsequent outcome in people with a given disease or health condition. Meta-analysis using individual participant data (IPD), where the raw data are synthesised from multiple studies, has been championed as the gold-standard for synthesising prognostic factor studies. We assessed the feasibility and conduct of this approach.

Methods

A systematic review to identify published IPD meta-analyses of prognostic factors studies, followed by detailed assessment of a random sample of 20 articles published from 2006. Six of these 20 articles were from the IMPACT (International Mission for Prognosis and Analysis of Clinical Trials in traumatic brain injury) collaboration, for which additional information was also used from simultaneously published companion papers.

Results

Forty-eight published IPD meta-analyses of prognostic factors were identified up to March 2009. Only three were published before 2000 but thereafter a median of four articles exist per year, with traumatic brain injury the most active research field. Availability of IPD offered many advantages, such as checking modelling assumptions; analysing variables on their continuous scale with the possibility of assessing for non-linear relationships; and obtaining results adjusted for other variables. However, researchers also faced many challenges, such as large cost and time required to obtain and clean IPD; unavailable IPD for some studies; different sets of prognostic factors in each study; and variability in study methods of measurement. The IMPACT initiative is a leading example, and had generally strong design, methodological and statistical standards. Elsewhere, standards are not always as high and improvements in the conduct of IPD meta-analyses of prognostic factor studies are often needed; in particular, continuous variables are often categorised without reason; publication bias and availability bias are rarely examined; and important methodological details and summary results are often inadequately reported.

Conclusions

IPD meta-analyses of prognostic factors are achievable and offer many advantages, as displayed most expertly by the IMPACT initiative. However such projects face numerous logistical and methodological obstacles, and their conduct and reporting can often be substantially improved.