Tufts PACE Clinical Predictive Model Registry: update 1990 through 2015

BACKGROUND

Clinical predictive models (CPMs) estimate the probability of clinical outcomes and hold the potential to improve decision-making and individualize care. The Tufts Predictive Analytics and Comparative Effectiveness (PACE) CPM Registry is a comprehensive database of cardiovascular disease (CVD) CPMs. The Registry was last updated in 2012, and there continues to be substantial growth in the number of available CPMs.

METHODS

We updated a systematic review of CPMs for CVD to include articles published from January 1990 to March 2015. CVD includes coronary artery disease (CAD), congestive heart failure (CHF), arrhythmias, stroke, venous thromboembolism (VTE), and peripheral vascular disease (PVD). The updated Registry characterizes CPMs based on population under study, model performance, covariates, and predicted outcomes.

RESULTS

The Registry includes 747 articles presenting 1083 models, including both prognostic (n = 1060) and diagnostic (n = 23) CPMs representing 183 distinct index condition/outcome pairs. There was a threefold increase in the number of CPMs published between 2005 and 2014, compared to the prior 10-year interval from 1995 to 2004. The majority of CPMs were derived from either North American (n = 455, 42%) or European (n = 344, 32%) populations. The database contains 265 CPMs predicting outcomes for patients with coronary artery disease, 196 CPMs for population samples at risk for incident CVD, and 158 models for patients with stroke. Approximately two thirds (n = 701, 65%) of CPMs report a c-statistic, with a median reported c-statistic of 0.77 (IQR, 0.05). Of the CPMs reporting validations, only 333 (57%) report some measure of model calibration. Reporting of discrimination but not calibration is improving over time (p for trend < 0.0001 and 0.39 respectively).

CONCLUSIONS

There is substantial redundancy of CPMs for a wide spectrum of CVD conditions. While the number of CPMs continues to increase, model performance is often inadequately reported and calibration is infrequently assessed. More work is needed to understand the potential impact of this literature.

Targeting Pioglitazone Hydrochloride Therapy After Stroke or Transient Ischemic Attack According to Pretreatment Risk for Stroke or Myocardial Infarction

Importance

There is growing recognition that patients may respond differently to therapy and that the average treatment effect from a clinical trial may not apply equally to all candidates for a therapy.

Objective

To determine whether, among patients with an ischemic stroke or transient ischemic attack and insulin resistance, those at higher risk for future stroke or myocardial infarction (MI) derive more benefit from the insulin-sensitizing drug pioglitazone hydrochloride compared with patients at lower risk.

Design, Setting, and Participants

A secondary analysis was conducted of the Insulin Resistance Intervention After Stroke trial, a double-blind, placebo-controlled trial of pioglitazone for secondary prevention. Patients were enrolled from 179 research sites in 7 countries from February 7, 2005, to January 15, 2013, and were followed up for a mean of 4.1 years through the study's end on July 28, 2015. Eligible participants had a qualifying ischemic stroke or transient ischemic attack within 180 days of entry and insulin resistance without type 1 or type 2 diabetes.

Interventions

Pioglitazone or matching placebo.

Main Outcomes and Measures

A Cox proportional hazards regression model was created using baseline features to stratify patients above or below the median risk for stroke or MI within 5 years. Within each stratum, the efficacy of pioglitazone for preventing stroke or MI was calculated. Safety outcomes were death, heart failure, weight gain, and bone fracture.

Results

Among 3876 participants (1338 women and 2538 men; mean [SD] age, 63 [11] years), the 5-year risk for stroke or MI was 6.0% in the pioglitazone group among patients at lower baseline risk compared with 7.9% in the placebo group (absolute risk difference, -1.9% [95% CI, -4.4% to 0.6%]). Among patients at higher risk, the risk was 14.7% in the pioglitazone group vs 19.6% for placebo (absolute risk difference, -4.9% [95% CI, -8.6% to 1.2%]). Hazard ratios were similar for patients below or above the median risk (0.77 vs 0.75; P = .92). Pioglitazone increased weight less among patients at higher risk but increased the risk for fracture more.

Conclusions and Relevance

After an ischemic stroke or transient ischemic attack, patients at higher risk for stroke or MI derive a greater absolute benefit from pioglitazone compared with patients at lower risk. However, the risk for fracture is also higher.

Trial Registration

clinicaltrials.gov Identifier: NCT00091949.

Regional Validation and Recalibration of Clinical Predictive Models for Patients With Acute Heart Failure

Background

Heart failure clinical practice guidelines recommend applying validated clinical predictive models (CPMs) to support decision making. While CPMs are now widely available, the generalizability of heart failure CPMs is largely unknown.

Methods and Results

We identified CPMs derived in North America that predict mortality for patients with acute heart failure and validated these models in different world regions to assess performance in a contemporary international clinical trial (N=4133) of patients with acute heart failure treated with guideline‐directed medical therapy. We performed independent external validations of 3 CPMs predicting in‐hospital mortality, 60‐day mortality, and 1‐year mortality, respectively. CPM discrimination decreased in all regional validation cohorts. The median change in area under the receiver operating curve was −0.09 (range −0.05 to −0.23). Regional calibration was highly variable (90th percentile of absolute difference between smoothed observed and predicted values range <1% to >50%). Calibration remained poor after global recalibrations; however, region‐specific recalibration procedures significantly improved regional performance (recalibrated 90th percentile of absolute difference range <1% to 5% across all regions and all models).

Conclusions

Acute heart failure CPM discrimination and calibration vary substantially across different world regions; region‐specific (as opposed to global) recalibration techniques are needed to improve CPM calibration.

Change in Testing, Awareness of Hemoglobin A1c Result, and Glycemic Control in US Adults, 2007-2014

This cross-sectional study uses NHANES survey data to examine trends in glycemic control and patient awareness of HbA_1c test results and targets between 2007 and 2014.

Comparison of the CHA2DS2-VASc, CHADS2, HAS-BLED, ORBIT, and ATRIA Risk Scores in Predicting Non-Vitamin K Antagonist Oral Anticoagulants-Associated Bleeding in Patients With Atrial Fibrillation

The increasing adoption of non-vitamin K antagonist oral anticoagulants (NOACs) for stroke prevention in atrial fibrillation (AF) necessitates a reassessment of bleeding risk scores. Because known risk factors for bleeding are largely the same as for stroke, we hypothesize that stroke risk scores could also be used to identify patients with high bleeding risks. We aimed to compare the performance of 2 stroke risk scores (Congestive Heart failure, hypertension, Age ≥75 [doubled], Diabetes, Stroke [doubled], Vascular disease, Age 65-74, and Sex [female] [CHA₂DS₂-VASc] and Cardiac failure, Hypertension, Age, Diabetes, Stroke [Doubled] [CHADS₂]) and 3 bleeding risk scores (hypertension, abnormal renal/liver function [1 point each], stroke, bleeding history or predisposition, labile INR, elderly [.65 years], drugs/alcohol concomitantly [1 point each] [HAS-BLED], Outcomes Registry for Better Informed Treatment of Atrial Fibrillation [ORBIT], and AnTicoagulation and Risk factors In Atrial fibrillation [ATRIA]) in predicting major and intracranial bleeding. Using a large US commercial insurance database, we identified 39,539 patients with nonvalvular AF who started NOACs between October 1, 2010 and June 30, 2015. The performance of risk scores was compared using C-statistic and net reclassification improvement (NRI). Over a total of 22,583 person-years, 665 patients (2.94% per year) had major bleeding, including 74 intracranial hemorrhages (0.33% per year). For the prediction of major bleeding, CHA₂DS₂-VASc had the highest C-statistic both as a continuous score (C-statistic 0.68) and as a categorical score (C-statistic 0.65). For the prediction of intracranial bleeding, CHADS₂ had the highest C-statistic both as a continuous score (C-statistic 0.66) and as a categorical score (C-statistic 0.66). There were no statistically significant differences between scores based on NRI. In conclusion, CHA₂DS₂-VASc, CHADS₂, HAS-BLED, ORBIT, and ATRIA had similar, albeit modest, performance in predicting NOAC-associated bleeding in patients with AF. Careful assessment and active management of bleeding risk factors may be warranted in all patients on NOACs who have high stroke risk scores.

Multistate Model to Predict Heart Failure Hospitalizations and All-Cause Mortality in Outpatients With Heart Failure With Reduced Ejection Fraction: Model Derivation and External Validation

BACKGROUND

Outpatients with heart failure (HF) who are at high risk for HF hospitalization and death may benefit from early identification. We sought to develop and externally validate a model to predict both HF hospitalization and mortality that accounts for the semicompeting nature of the 2 outcomes and captures the risk associated with the transition from the stable outpatient state to the post-HF hospitalization state.

METHODS AND RESULTS

A multistate model to predict HF hospitalization and all-cause mortality was derived using data (n=3834) from the HEAAL study (Heart Failure Endpoint evaluation of Angiotensin II Antagonist Losartan), a multinational randomized trial in symptomatic patients with reduced left ventricular ejection fraction. Twelve easily and reliably obtainable demographic and clinical predictors were prespecified for model inclusion. Model performance was assessed in the SCD-HeFT cohort (Sudden Cardiac Death in Heart Failure Trial; n=2521). At 1 year, the probability of being alive without HF hospitalization was 94% for a typical patient in the lowest risk quintile and 77% for a typical patient in the highest risk quintile and this variability in risk continued through 7 years of follow-up. The model c-index was 0.72 in the derivation cohort, 0.66 in the validation cohort, and 0.69 in the implantable cardiac defibrillator arm of the validation cohort. There was excellent calibration across quintiles of predicted risk.

CONCLUSIONS

Our findings illustrate the advantages of a multistate modeling approach, providing estimates of HF hospitalization and death in the same model, comparison of predictors for the different outcomes and demonstrating the different trajectories of patients based on baseline characteristics and intermediary events.

CLINICAL TRIAL REGISTRATION

URL: http://www.clinicaltrials.gov. Unique identifiers: NCT00000609 and NCT00090259.

Understanding the Value of Individualized Information: The Impact of Poor Calibration or Discrimination in Outcome Prediction Models

BACKGROUND

Risk prediction models allow for the incorporation of individualized risk and clinical effectiveness information to identify patients for whom therapy is most appropriate and cost-effective. This approach has the potential to identify inefficient (or harmful) care in subgroups at different risks, even when the overall results appear favorable. Here, we explore the value of personalized risk information and the factors that influence it.

METHODS

Using an expected value of individualized care (EVIC) framework, which monetizes the value of customizing care, we developed a general approach to calculate individualized incremental cost effectiveness ratios (ICERs) as a function of individual outcome risk. For a case study (tPA v. streptokinase to treat possible myocardial infarction), we used a simulation to explore how an EVIC is influenced by population outcome prevalence, model discrimination (c-statistic) and calibration, and willingness-to-pay (WTP) thresholds.

RESULTS

In our simulations, for well-calibrated models, which do not over- or underestimate predicted v. observed event risk, the EVIC ranged from $0 to $700 per person, with better discrimination (higher c-statistic values) yielding progressively higher EVIC values. For miscalibrated models, the EVIC ranged from -$600 to $600 in different simulated scenarios. The EVIC values decreased as discrimination improved from a c-statistic of 0.5 to 0.6, before becoming positive as the c-statistic reached values of ~0.8.

CONCLUSIONS

Individualizing treatment decisions using risk may produce substantial value but also has the potential for net harm. Good model calibration ensures a non-negative EVIC. Improvements in discrimination generally increase the EVIC; however, when models are miscalibrated, greater discriminating power can paradoxically reduce the EVIC under some circumstances.

Biases in Individualized Cost-effectiveness Analysis: Influence of Choices in Modeling Short-Term, Trial-Based, Mortality Risk Reduction and Post-Trial Life Expectancy

BACKGROUND

The benefits and costs of a treatment are typically heterogeneous across individual patients. Randomized clinical trials permit the examination of individualized treatment benefits over the trial horizon but extrapolation to lifetime horizon usually involves combining trial-based individualized estimates of short-term risk reduction with less detailed (less granular) population life tables. However, the underlying assumption of equal post-trial life expectancy for low- and high-risk patients of the same sex and age is unrealistic. We aimed to study the influence of unequal granularity between models of short-term risk reduction and life expectancy on individualized estimates of cost-effectiveness of aggressive thrombolysis for patients with an acute myocardial infarction.

METHODS

To estimate life years gained, we multiplied individualized estimates of short-term risk reduction either with less granular and with equally granular post-trial life expectancy estimates. Estimates of short-term risk reduction were obtained from GUSTO trial data (30,510 patients) using logistic regression analysis with treatment, sex, and age as predictor variables. Life expectancy estimates were derived from sex- and age-specific US life tables.

RESULTS

Based on sex- and age-specific, short-term risk reductions but average population life expectancy (less granularity), we found that aggressive thrombolysis was cost-effective (incremental cost-effectiveness ratio below $50,000) for women above age 49 y and men above age 53 y (92% and 69% of the population, respectively). Considering sex- and age-specific short-term mortality risk reduction and correspondingly sex- and age-specific life expectancy (equal granularity), aggressive thrombolysis was cost-effective for men above age 45 y and women above age 50 y (95% and 76% of the population, respectively).

CONCLUSIONS

Failure to model short-term risk reduction and life expectancy at an equal level of granularity may bias our estimates of individualized cost-effectiveness and misallocate resources.

Abstract 129: Clinical Predictive Models for Valvular Heart Disease: A Systematic Review of the Literature

Objectives: Pre-procedure risk assessment is central to clinical decision making for patients with advanced valvular heart disease (VHD) and treatments are increasingly being offered to patients with elevated pre-procedure risk. While there are numerous clinical predictive models (CPMs) available for patients with VHD, the relative performance of these CPMs is largely unknown. Here we describe the performance of CPMs available for patients with VHD with specific attention to whether CPMs have been externally validated.

Methods: To identify CPMs for patients with VHD, we conducted a systematic review of the Tufts PACE CPM Registry, a comprehensive database of cardiovascular CPMs. For each identified CPM for patients with VHD, we performed a complete citation search using Scopus to identify any external validations of these models published in other articles. We extracted information on CPM performance in both the original report and also the external validations. For external validations we calculated the relative percent decrease in discrimination.

Results: We identified 41 CPMs predicting outcomes for patients with VHD. 33 (81%) predict outcomes following surgical intervention, 5 (12%) predict outcomes following percutaneous interventions, and 3 (7%) predict outcomes in the absence of intervention. Only 30/41 (73%) of the CPMs report a c-statistic. The median reported c- statistic was 0.77 [IQR, 0.04] for CPMs predicting outcomes following surgical interventions, 0.68 [IQR, 0.04] for CPMs for percutaneous interventions, and 0.83 [IQR, 0.07] for CPMs predicting outcomes in the absence of intervention. While a total of 69 external validations of these CPMs have been published, only 21 (51%) of the CPMs have ever been externally validated. For external validations that report c- statistics, we noted a median percent decrement in discrimination of -27.6% [IQR, -37.4] ( Figure) .

Conclusion: While there are numerous CPMs for patients with VHD, performance is often incompletely reported and half of these CPMs have never been externally validated. The CPMs that have been externally validated generally show substantially worse discrimination in external datasets compared to the derivation datasets.

Abstract 239: A Field Synopsis of the Role of Race in Clinical Prediction Models for Cardiovascular Disease

Introduction: Race and ethnicity-based differences and disparities have been noted in cardiovascular disease (CVD), including inequalities in access to care and differences in genetics and pharmacokinetics that affect response to treatment and development of clinical outcomes. Risk scores commonly used in CVD prevention, such as the Pooled Cohort Equation, include terms for race, yet to date there has been no systematic evaluation of the role of race and ethnicity in clinical prediction models (CPMs). To better understand the potential impact of race-specific clinical decisions encouraged by these models, we conducted a field synopsis of the role of race and ethnicity in CPMs for CVD.

Methods: We identified CPMs in the Tufts PACE CPM Database, a systematic review of CVD-related CPMs published from 1/1990-3/2015. We report the proportion of models including the effect of race or ethnicity on CVD incidence or prognosis, summarize the directionality of the effects of race (harm or protection), and explore factors influencing the inclusion of race.

Results: Out of 908 CPMs with CVD as either an index condition or outcome, only 24 (3%) contained a coefficient for race, ethnicity or a combination of race and ethnicity, or presented race-stratified equations. Among the overall group of CVD-related models, the racial or ethnic composition of the underlying model development cohort was reported for only 1 out of every 5 models (168/908 models (19%)), but was reported in the majority of models that included race (22/24 (92%)). Among models where the racial composition of the cohort was reported, the proportion of patients of white race was the same (79%) in models that included race vs. excluded race. The inclusion of race/ethnicity as a covariate or stratification variable was more common in models predicting incident CVD versus prognosis for patients with known CVD (9/126 (7%) versus 15/782 (2%), respectively, p<0.001)). Race/ethnicity was included infrequently even in models predicting outcomes in patient populations where racial differences have been documented: heart failure-mortality (2%, 2/107), population sample-CVD (4%, 2/52), and population sample-stroke (0%, 0/26). Among the 23 models that contained a coefficient for race/ethnicity, 11 (48%) indicated a higher risk of the outcome for patients of non-white race, while 12 (52%) indicated higher risk for patients of white race.

Conclusions: Race/ethnicity is rarely included in CVD-related CPMs, although racial and ethnic differences in outcome risk have been documented for many conditions, and several commonly used CPMs include race. Model development from cohorts including greater numbers of non-white patients may uncover additional or more consistent racial/ethnic effects.

Abstract 130: The Tufts PACE Clinical Predictive Model Registry: Update 1990 Through 2015

Background: Clinical Predictive Models (CPMs) estimate the probability of clinical outcomes and hold the potential to improve decision making and individualize care. The Tufts Predictive Analytics and Comparative Effectiveness (PACE) CPM Registry is a comprehensive database of cardiovascular (CVD) CPMs. The Registry was last updated in 2012 and there has been substantial growth in the number of CPMs that are available.

Methods and Results: We updated a systematic review of CPMs for CVD to include articles published from January 1990 to March 2015. The Registry now includes prognostic (n=1047) and diagnostic (n = 27) CPMs. There was a 3-fold increase in the number of CPMs published between 2005 and 2014, when compared to 1995 and 2004 ( Figure) . There are 1074 models included in this database representing 68 distinct index/ outcome (I/O) pairings. 792 (72%) of the CPMs were derived from either North American (n = 448) or European (n = 344) populations. The database contains 265 CPMs predicting outcomes for patients with coronary artery disease, 187 CPMs for population samples at risk for incident CVD, and 158 models for patients with prior stroke. 697 (65%) CPMs report a c- statistic and overall the median reported c- statistic was 0.77 [IQR, 0.09]. Of the 10 most common index conditions, discrimination was highest for CPMs predicting outcomes following cardiac arrest (14/27 reporting, median c- statistic 0.83 [IQR, 0.08]). Discrimination was lowest for CPMs predicting outcomes for patients with other types of arrhythmias (16/22 reporting, median c -statistic 0.71 [IQR 0.07]). Of the CPMs included in this Registry only 422 (39%) report some measure of model calibration.

Conclusions: There is continued growth and substantial redundancy of CPMs for a wide spectrum of CVD conditions. While the number of CPMs continues to increase, model performance is often inadequately reported and calibration is infrequently assessed. More work is needed to understand the potential impact of this literature.

Abstract 73: Risk of Paradoxical Embolism (RoPE) Score Stratification of Pooled Pfo Closure Clinical Trial Data: Lack of Evidence for Improvement in Patient Selection for Closure

Introduction: The Risk of Paradoxical Embolism (RoPE) Score can disaggregate patients with cryptogenic stroke (CS) and patent foramen ovale (PFO) into those who are more likely to have a pathogenic PFO (high RoPE score) than an incidental PFO (low RoPE score). Those with higher RoPE scores have a lower risk of recurrent stroke and different recurrence predictors compared to those with low RoPE scores.

Hypotheses: Patients with high RoPE scores benefit more from PFO closure than patients with low RoPE scores. Patients with high RoPE scores and risk factors for recurrence should benefit even more.

Methods: The RoPE score was created from a database of CS patients with known PFO status to estimate stratum-specific PFO-attributable fraction and recurrence rates. Variables that predict stroke recurrence in high RoPE score groups (atrial septal aneurysm, history of stroke/TIA prior to index event) were added – the “RoPE Recurrence Score.” Using pooled individual patient data from all 3 RCTs of PFO closure vs. medical therapy (ITT populations; stroke outcome) we tested the ability of the scores to predict the heterogeneity of response to assigned treatment.

Results: The mean RoPE score was significantly higher (6.8 vs. 6.3) with smaller variance (Stdev 1.5 vs. 1.9) in the pooled RCT population than in the original RoPE cohort (p<0.0001). Hazard ratios favoring closure were 0.82 (0.42-1.59, p=0.56) in the low RoPE score (<7) group and 0.31 (0.11-0.85, p=0.02) in the high RoPE score (≥7) group but the interaction p-value was not significant (p=0.12). The RoPE Recurrence score did not improve the prediction of treatment response (low score HR=0.65 (0.31-1.37), p=0.26; high score HR=0.58 (0.26-1.26), p=0.17; interaction p=0.82).

Conclusion: As expected, the HR favoring closure trended lower in the high RoPE score group in the RCTs but missed statistical significance. The RoPE Recurrence score did not improve the prediction. This may be due to the narrow distribution of relatively high RoPE scores amongst RCT patients and so a low power to detect heterogeneity of treatment effect. Given that PFO closure can only prevent PFO-related recurrences, the treatment effect may also have been obscured by including recurrent strokes with non-PFO-related mechanisms.

Survival Analyses and Prognosis of Plasma-Cell Myeloma and Plasmacytoma-Like Posttransplantation Lymphoproliferative Disorders

BACKGROUND

Multiple myeloma/plasmacytoma-like posttransplantation lymphoproliferative disorder (PTLD-MM) is a rare complication of solid organ transplantation. Case series have shown variable outcomes, and survival data in the modern era are lacking.

PATIENTS AND METHODS

A cohort of 212 PTLD-MM patients was identified in the Scientific Registry of Transplant Recipients between 1999 and 2011. Overall survival (OS) was estimated by the Kaplan-Meier method, and the effects of treatment and patient characteristics on OS were evaluated by Cox proportional hazards models. OS in 185 PTLD-MM patients was compared to 4048 matched controls with multiple myeloma (SEER-MM) derived from Surveillance, Epidemiology, and End Results (SEER) data.

RESULTS

Men comprised 71% of patients; extramedullary disease was noted in 58%. Novel therapeutic agents were used in 19% of patients (more commonly during 2007-2011 vs. 1999-2006; P = .01), reduced immunosuppression in 55%, and chemotherapy in 32%. Median OS was 2.4 years and improved in the later time period (adjusted hazard ratio [aHR], 0.64, P = .05). Advanced age, creatinine > 2 g/dL, white race, and use of OKT3 were associated with inferior OS in multivariable analysis. OS of PTLD-MM patients is significantly inferior to SEER-MM patients (aHR, 1.6, P < .001). Improvements in OS over time differed between PTLD-MM and SEER-MM. Median OS of patients diagnosed from 2000 to 2005 was shorter for PTLD-MM than SEER-MM patients (18 vs. 47 months, P < .001). There was no difference among those diagnosed from 2006 to 2010 (44 months vs. median not reached, P = .5; interaction P = .08).

CONCLUSION

Age at diagnosis, elevated creatinine, white race, and OKT3 were associated with inferior survival in patients with PTLD-MM. Survival of PTLD-MM is inferior to SEER-MM, although significant improvements in survival have been documented.

Risk and treatment effect heterogeneity: re-analysis of individual participant data from 32 large clinical trials

Background

Risk of the outcome is a mathematical determinant of the absolute treatment benefit of an intervention, yet this can vary substantially within a trial population, complicating the interpretation of trial results.

Methods

We developed risk models using Cox or logistic regression on a set of large publicly available randomized controlled trials (RCTs). We evaluated risk heterogeneity using the extreme quartile risk ratio (EQRR, the ratio of outcome rates in the lowest risk quartile to that in the highest) and skewness using the median to mean risk ratio (MMRR, the ratio of risk in the median risk patient to the average). We also examined heterogeneity of treatment effects (HTE) across risk strata.

Results

We describe 39 analyses using data from 32 large trials, with event rates across studies ranging from 3% to 63% (median = 15%, 25th-75th percentile = 9-29%). C-statistics of risk models ranged from 0.59 to 0.89 (median = 0.70, 25th-75th percentile = 0.65-0.71). The EQRR ranged from 1.8 to 50.7 (median = 4.3, 25th-75th percentile = 3.0-6.1). The MMRR ranged from 0.4 to 1.0 (median = 0.86, 25th-75th percentile = 0.80-0.92). EQRRs were predictably higher and MMRRs predictably lower as the c-statistic increased or the overall outcome incidence decreased. Among 18 comparisons with a significant overall treatment effect, there was a significant interaction between treatment and baseline risk on the proportional scale in only one. The difference in the absolute risk reduction between extreme risk quartiles ranged from -3.2 to 28.3% (median = 5.1%; 25th-75th percentile = 0.3-10.9).

Conclusions

There is typically substantial variation in outcome risk in clinical trials, commonly leading to clinically significant differences in absolute treatment effects Most patients have outcome risks lower than the trial average reflected in the summary result. Risk-stratified trial analyses are feasible and may be clinically informative, particularly when the outcome is predictable and uncommon.

Development and Validation of a Predictive Model for Short- and Medium-Term Hospital Readmission Following Heart Valve Surgery

Background

Although models exist for predicting hospital readmission after coronary artery bypass surgery, no such models exist for predicting readmission after heart valve surgery (HVS).

Methods and Results

Using a geographically and structurally diverse sample of US hospitals (Premier Inpatient Database, January 2007–June 2011), we examined patient, hospital, and clinical factors predictive of short‐ and medium‐term hospital readmission post‐HVS. We set aside 20% of hospitals for model validation. A generalized estimating equation model accounted for clustering within hospitals. At 219 hospitals, we identified 38 532 patients (67 years, 56% male, 62% aortic valve surgery) who underwent HVS. A total of 3125 (7.8%) and 4943 (12.8%) patients were readmitted to the index hospital within 1 and 3 months, respectively. Our 3‐month model predicted readmission rates between 3% and 61% with fair discrimination (C‐statistic, 0.67) and good calibration (predicted vs observed differences in validation cohort averaged 1.9% across all deciles of predicted readmission risk). Results were similar for our 1‐month model and our simplified 3‐month model (suitable for clinical use), which used the 5 strongest predictors of readmission: transfused units of packed Red blood cells, presence of End‐stage renal disease, type of Valve surgery, Emergency hospital admission, and hospital Length of stay (REVEaL).

Conclusions

We described and validated key factors that predict short‐ and medium‐term hospital readmission post‐HVS. These models should enable clinicians to identify individuals with HVS who are at increased risk for hospital readmission and are most likely to benefit from improved postdischarge care and follow‐up.

Practice advisory: Recurrent stroke with patent foramen ovale (update of practice parameter): Report of the Guideline Development, Dissemination, and Implementation Subcommittee of the American Academy of Neurology

OBJECTIVE

To update the 2004 American Academy of Neurology guideline for patients with stroke and patent foramen ovale (PFO) by addressing whether (1) percutaneous closure of PFO is superior to medical therapy alone and (2) anticoagulation is superior to antiplatelet therapy for the prevention of recurrent stroke.

METHODS

Systematic review of the literature and structured formulation of recommendations.

CONCLUSIONS

Percutaneous PFO closure with the STARFlex device possibly does not provide a benefit in preventing stroke vs medical therapy alone (risk difference [RD] 0.13%, 95% confidence interval [CI] -2.2% to 2.0%). Percutaneous PFO closure with the AMPLATZER PFO Occluder possibly decreases the risk of recurrent stroke (RD -1.68%, 95% CI -3.18% to -0.19%), possibly increases the risk of new-onset atrial fibrillation (AF) (RD 1.64%, 95% CI 0.07%-3.2%), and is highly likely to be associated with a procedural complication risk of 3.4% (95% CI 2.3%-5%). There is insufficient evidence to determine the efficacy of anticoagulation compared with antiplatelet therapy in preventing recurrent stroke (RD 2%, 95% CI -21% to 25%).

RECOMMENDATIONS

Clinicians should not routinely offer percutaneous PFO closure to patients with cryptogenic ischemic stroke outside of a research setting (Level R). In rare circumstances, such as recurrent strokes despite adequate medical therapy with no other mechanism identified, clinicians may offer the AMPLATZER PFO Occluder if it is available (Level C). In the absence of another indication for anticoagulation, clinicians may routinely offer antiplatelet medications instead of anticoagulation to patients with cryptogenic stroke and PFO (Level C).

Field Synopsis of the Role of Sex in Stroke Prediction Models

Background

Guidelines for stroke prevention recommend development of sex‐specific stroke risk scores. Incorporating sex in Clinical Prediction Models (CPMs) may support sex‐specific clinical decision making. To better understand their potential to guide sex‐specific care, we conducted a field synopsis of the role of sex in stroke‐related CPMs.

Methods and Results

We identified stroke‐related CPMs in the Tufts Predictive Analytics and Comparative Effectiveness CPM Database, a systematic summary of cardiovascular CPMs published from January 1990 to May 2012. We report the proportion of models including the effect of sex on stroke incidence or prognosis, summarize the directionality of the predictive effects of sex, and explore factors influencing the inclusion of sex. Of 92 stroke‐related CPMs, 30 (33%) contained a coefficient for sex or presented sex‐stratified models. Only 12/58 (21%) CPMs predicting outcomes in patients included sex, compared to 18/30 (60%) models predicting first stroke (P<0.0001). Sex was most commonly included in models predicting stroke among a general population (69%). Female sex was consistently associated with reduced mortality after ischemic stroke (n=4) and higher risk of stroke from arrhythmias or coronary revascularization (n=5). Models predicting first stroke versus outcomes among patients with stroke (odds ratio=5.75, 95% CI 2.18–15.14, P<0.001) and those developed from larger versus smaller sample sizes (odds ratio=4.58, 95% CI 1.73–12.13, P=0.002) were significantly more likely to include sex.

Conclusions

Sex is included in a minority of published CPMs, but more frequently in models predicting incidence of first stroke. The importance of sex‐specific care may be especially well established for primary prevention.

Abstract 107: A Field Synopsis of Sex in Clinical Prediction Models for Cardiovascular Disease

Introduction: Incorporating sex in Clinical Prediction Models (CPMs) may support sex-specific clinical decision making. Risk scores commonly used in CVD prevention, such as the Pooled Cohort Equation and the Framingham risk score for 10 year CVD risk, present sex-specific algorithms, yet to date, there has been no systematic summary of the role of sex across CPMs. To better understand the potential influence these models might have on sex-specific care, we conducted a field synopsis of the role of sex in CPMs for CVD.

Methods: We identified CPMs in the Tufts PACE CPM Database, a systematic review of CVD CPMs published from 1/1990-5/2012. We report the proportion of models including the effect of sex on CVD incidence or prognosis, summarize the directionality of sex effects (harm or protection associated with female sex), and explore factors influencing the inclusion of sex.

Results: Out of 592 CPMs with CVD as either an index condition or outcome, 173 (34%) contained a coefficient for sex and 27 (5%) presented sex-stratified models. Sex was over 2.5 times more likely to be included in models predicting CVD incidence in a general population sample versus models predicting prognostic outcomes among patients with known CVD (79% (54/68) vs. 29% (146/498), p<0.0001). Among the 366 CVD-related models that did not include sex as a covariate or stratification variable, 71% reported that sex had been considered as a candidate for inclusion based on clinical or statistical criteria. Being a woman was associated with lower risk of death in 8 of 8 models predicting mortality among patients with heart failure that included sex as a covariate (see figure), yet a higher risk of death among women undergoing revascularization procedures in 10 of 12 CPMs. In multivariable analysis, the number of outcome events (OR=2.6, 95% CI 1.6-4.4, p=0.0002) and a cohort defined as a population sample at risk for developing CVD (OR=6.2, 95% CI 2.7-14.1, p<0.0001) were significantly associated with inclusion of sex in CPMs.

Conclusions: Sex is included in about one third of published CPMs, but more frequently in models predicting incidence of CVD. The importance of sex-specific care may be especially well established for primary prevention. The rapidly growing literature on CPMs may yield important insights to guide sex-specific CVD prevention and treatment.

Field Synopsis of Sex in Clinical Prediction Models for Cardiovascular Disease

BACKGROUND

Several widely used risk scores for cardiovascular disease (CVD) incorporate sex effects, yet there has been no systematic summary of the role of sex in clinical prediction models (CPMs). To better understand the potential of these models to support sex-specific care, we conducted a field synopsis of sex effects in CPMs for CVD.

METHODS AND RESULTS

We identified CPMs in the Tufts Predictive Analytics and Comparative Effectiveness CPM Registry, a comprehensive database of CVD CPMs published from January 1990 to May 2012. We report the proportion of models including sex effects on CVD incidence or prognosis, summarize the directionality of the predictive effects of sex, and explore factors influencing the inclusion of sex. Of 592 CVD-related CPMs, 193 (33%) included sex as a predictor or presented sex-stratified models. Sex effects were included in 78% (53/68) of models predicting incidence of CVD in a general population, versus only 35% (59/171), 21% (12/58), and 17% (12/72) of models predicting outcomes in patients with coronary artery disease, stroke, and heart failure, respectively. Among sex-including CPMs, women with heart failure were at lower mortality risk in 8 of 8 models; women undergoing revascularization for coronary artery disease were at higher mortality risk in 10 of 12 models. Factors associated with the inclusion of sex effects included the number of outcome events and using cohorts at-risk for CVD (rather than with established CVD).

CONCLUSIONS

Although CPMs hold promise for supporting sex-specific decision making in CVD clinical care, sex effects are included in only one third of published CPMs.

Hospitals' Patterns of Use of Noninvasive Ventilation in Patients With Asthma Exacerbation

BACKGROUND

Limited data are available on the use of noninvasive ventilation in patients with asthma exacerbations. The objective of this study was to characterize hospital patterns of noninvasive ventilation use in patients with asthma and to evaluate the association with the use of invasive mechanical ventilation and case fatality rate.

METHODS

This cross-sectional study used an electronic medical record dataset, which includes comprehensive pharmacy and laboratory results from 58 hospitals. Data on 13,558 patients admitted from 2009 to 2012 were analyzed. Initial noninvasive ventilation (NIV) or invasive mechanical ventilation (IMV) was defined as the first ventilation method during hospitalization. Hospital-level risk-standardized rates of NIV among all admissions with asthma were calculated by using a hierarchical regression model. Hospitals were grouped into quartiles of NIV to compare the outcomes.

RESULTS

Overall, 90.3% of patients with asthma were not ventilated, 4.0% were ventilated with NIV, and 5.7% were ventilated with IMV. Twenty-two (38%) hospitals did not use NIV for any included admissions. Hospital-level adjusted NIV rates varied considerably (range, 0.4-33.1; median, 5.2%). Hospitals in the highest quartile of NIV did not have lower IMV use (5.4% vs 5.7%), but they did have a small but significantly shorter length of stay. Higher NIV rates were not associated with lower risk-adjusted case fatality rates.

CONCLUSIONS

Large variation exists in hospital use of NIV for patients with an acute exacerbation of asthma. Higher hospital rates of NIV use does not seem to be associated with lower IMV rates. These results indicate a need to understand contextual and organizational factors contributing to this variability.