Pennsylvania's Focus on Heart Attack--grading the scorecard

Do Cardiac Surgery Report Cards Reduce Mortality? Assessing the Evidence

Provider report cards feature prominently in ongoing efforts to improve patient quality. A well-known example is the cardiac surgery report-card program started in New York, which publicly compares hospital and surgeon performance. Public report cards have been associated with decreases in cardiac surgery mortality, but there is substantial disagreement over the source(s) of the improvement. This article develops a conceptual framework to explain how report-card-related responses could result in lower mortality and reviews the evidence. Existing research shows that report cards have not greatly changed referral patterns. How much providers increased their quality of care and altered their selection of patients remains unresolved, and alternative explanations have not been well studied. Future research should expand the number of states and years covered and exploit the variation in institutional features to improve our understanding of the relationship between report cards and outcomes.

Comparing hospital mortality--how to count does matter for patients hospitalized for acute myocardial infarction (AMI), stroke and hip fracture

Background

Mortality is a widely used, but often criticised, quality indicator for hospitals. In many countries, mortality is calculated from in-hospital deaths, due to limited access to follow-up data on patients transferred between hospitals and on discharged patients. The objectives were to: i) summarize time, place and cause of death for first time acute myocardial infarction (AMI), stroke and hip fracture, ii) compare case-mix adjusted 30-day mortality measures based on in-hospital deaths and in-and-out-of hospital deaths, with and without patients transferred to other hospitals.

Methods

Norwegian hospital data within a 5-year period were merged with information from official registers. Mortality based on in-and-out-of-hospital deaths, weighted according to length of stay at each hospital for transferred patients (W30D), was compared to a) mortality based on in-and-out-of-hospital deaths excluding patients treated at two or more hospitals (S30D), and b) mortality based on in-hospital deaths (IH30D). Adjusted mortalities were estimated by logistic regression which, in addition to hospital, included age, sex and stage of disease. The hospitals were assigned outlier status according to the Z-values for hospitals in the models; low mortality: Z-values below the 5-percentile, high mortality: Z-values above the 95-percentile, medium mortality: remaining hospitals.

Results

The data included 48 048 AMI patients, 47 854 stroke patients and 40 142 hip fracture patients from 55, 59 and 58 hospitals, respectively. The overall relative frequencies of deaths within 30 days were 19.1% (AMI), 17.6% (stroke) and 7.8% (hip fracture). The cause of death diagnoses included the referral diagnosis for 73.8-89.6% of the deaths within 30 days. When comparing S30D versus W30D outlier status changed for 14.6% (AMI), 15.3% (stroke) and 36.2% (hip fracture) of the hospitals. For IH30D compared to W30D outlier status changed for 18.2% (AMI), 25.4% (stroke) and 27.6% (hip fracture) of the hospitals.

Conclusions

Mortality measures based on in-hospital deaths alone, or measures excluding admissions for transferred patients, can be misleading as indicators of hospital performance. We propose to attribute the outcome to all hospitals by fraction of time spent in each hospital for patients transferred between hospitals to reduce bias due to double counting or exclusion of hospital stays.

The impact of percutaneous coronary interventions on outcomes for acute myocardial infarction in Pennsylvania

This research estimates the benefits associated with percutaneous coronary interventions (PCIs) for patients with acute myocardial infarction (AMI) treated at hospitals in Pennsylvania. We studied 31 351 patients with AMI in Pennsylvania during the year 2000, including 10 170 who received PCI. Univariate comparisons between groups were made using chi2 tests for categorical outcomes and Student's t tests for continuous outcomes. A logit model for proportions was used to model the relationship between mortality and the proportion of AMI patients who received PCI. The mortality rate for patients undergoing PCI was significantly lower than for those being treated medically (1.4% vs 15.8%, P<.0001). Furthermore, significant survival benefits associated with PCI persisted when patients were stratified by age, sex, type of infarction, and severity at admission. At the hospital level, higher rates of PCI were associated with a significantly lower overall mortality rate among patients with AMI (P<.0001).

Comparison of Clinical and Administrative Data Sources for Hospital Coronary Artery Bypass Graft Surgery Report Cards

Background— Regardless of statistical methodology, public performance report cards must use the highest-quality validated data, preferably from a prospectively maintained clinical database. Using logistic regression and hierarchical models, we compared hospital cardiac surgery profiling results based on clinical data with those derived from contemporaneous administrative data.

Methods and Results— Fiscal year 2003 isolated coronary artery bypass grafting surgery results based on an audited and validated Massachusetts clinical registry were compared with those derived from a contemporaneous state administrative database, the latter using the inclusion/exclusion criteria and risk model of the Agency for Healthcare Research and Quality. There was a 27.4% disparity in isolated coronary artery bypass grafting surgery volume (4440 clinical, 5657 administrative), a 0.83% difference in observed in-hospital mortality (2.05% versus 2.88%), corresponding differences in risk-adjusted mortality calculated by various statistical methodologies, and 1 hospital classified as an outlier only with the administrative data–based approach. The discrepancies in volumes and risk-adjusted mortality were most notable for higher-volume programs that presumably perform a higher proportion of combined procedures that were misclassified as isolated coronary artery bypass grafting surgery in the administrative cohort. Subsequent analyses of a patient cohort common to both databases revealed the smoothing effect of hierarchical models, a 9% relative difference in mortality (2.21% versus 2.03%) resulting from nonstandardized mortality end points, and 1 hospital classified as an outlier using logistic regression but not using hierarchical regression.

Conclusions— Cardiac surgery report cards using administrative data are problematic compared with those derived from audited and validated clinical data, primarily because of case misclassification and nonstandardized end points.

Measuring Performance For Treating Heart Attacks And Heart Failure: The Case For Outcomes Measurement

To complement the current process measures for treating patients with heart attacks and with heart failure, which target gaps in quality but do not capture patient outcomes, the Centers for Medicare and Medicaid Services (CMS) has proposed the public reporting of hospital-level thirty-day mortality for these conditions in 2007. We present the case for including measurements of outcomes in the assessment of hospital performance, focusing on the care of patients with heart attacks and with heart failure. Recent developments in the methodology and standards for outcomes measurement have laid the groundwork for incorporating outcomes into performance monitoring efforts for these conditions.

Accuracy of hospital report cards based on administrative data

CONTEXT

Many of the publicly available health quality report cards are based on administrative data. ICD-9-CM codes in administrative data are not date stamped to distinguish between medical conditions present at the time of hospital admission and complications, which occur after hospital admission. Treating complications as preexisting conditions gives poor-performing hospitals "credit" for their complications and may cause some hospitals that are delivering low-quality care to be misclassified as average- or high-performing hospitals.

OBJECTIVE

To determine whether hospital quality assessment based on administrative data is impacted by the inclusion of condition present at admission (CPAA) modifiers in administrative data as a date stamp indicator.

DESIGN, SETTING, AND PATIENTS

Retrospective cohort study based on 648,866 inpatient admissions between 1998 and 2000 for coronary artery bypass graft (CABG) surgery, coronary angioplasty (PTCA), carotid endarterectomy (CEA), abdominal aortic aneurysm (AAA) repair, total hip replacement (THR), acute MI (AMI), and stroke using the California State Inpatient Database which includes CPAA modifiers. Hierarchical logistic regression was used to create separate condition-specific risk adjustment models. For each study population, one model was constructed using only secondary diagnoses present at admission based on the CPAA modifier: "date stamp" model. The second model was constructed using all secondary diagnoses, ignoring the information present in the CPAA modifier: the "no date stamp model." Hospital quality was assessed separately using the "date stamp" and the "no date stamp" risk-adjustment models.

RESULTS

Forty percent of the CABG hospitals, 33 percent of the PTCA hospitals, 40 percent of the THR hospitals, and 33 percent of the AMI hospitals identified as low-performance hospitals by the "date stamp" models were not classified as low-performance hospitals by the "no date stamp" models. Fifty percent of the CABG hospitals, 33 percent of the PTCA hospitals, 50 percent of the CEA hospitals, and 36 percent of the AMI hospitals identified as low-performance hospitals by the "no date stamp" models were not identified as low-performance hospitals by the "date stamp" models. The inclusion of the CPAA modifier had a minor impact on hospital quality assessment for AAA repair, stroke, and CEA.

CONCLUSION

This study supports the hypothesis that the use of routine administrative data without date stamp information to construct hospital quality report cards may result in the mis-identification of hospital quality outliers. However, the CPAA modifier will need to be further validated before date stamped administrative data can be used as the basis for health quality report cards.

Ranking Hospitals According to Acute Myocardial Infarction Mortality

OBJECTIVE

The objective of this population-based observational cohort study was to estimate the extent to which the inclusion/exclusion of transferred patients with acute myocardial infarction (AMI) impacts on hospital performance rankings.

SUBJECTS

The authors studied 91,633 adult patients admitted to 116 acute care hospitals in Quebec, Canada, with a primary diagnosis of AMI between 1992 and 1999.

MAIN OUTCOME MEASURE

Hospital performance ranks, based on 30-day AMI mortality rates, were estimated with hierarchical models and compared using 3 different methods for handling transferred patients (exclude all transfers; include transfers and assign outcome to the referring hospital; include transfers and assign outcome to the receiving hospital). The explanatory variable of interest was the hospital to which the patient's outcome was attributed.

RESULTS

Using the 3 methods, 4 hospitals were ranked "best performers" once, and 1 hospital ranked among the best in 2 of the 3 analyses performed. Nine hospitals were ranked "worst performers" at least once (4 of which ranked among the "worst" once only, 2 ranked among the "worst" twice, and 3 were consistently ranked "worst performers" in all analyses). There was significant variation in mortality rates among hospitals, and the difference in the rates between the highest and lowest ranking hospitals exceeded the clinically relevant benchmark of 1%.

CONCLUSIONS

Performance evaluation studies that compare hospital mortality rates typically exclude transferred patients. However, methods used to deal with AMI patient transfers influenced hospital ranks when comparing 30-day mortality rates. Excluding transfers may lead to an inaccurate depiction of the quality of healthcare services in regionalized healthcare systems that call for the timely interhospital transfer of patients with AMI.

Does date stamping ICD-9-CM codes increase the value of clinical information in administrative data?

CONTEXT

Comorbidity measures are designed to exclude complications when they map International Classification of Diseases (ICD-9-CM) codes to diagnostic categories. The use of data fields that indicates whether each secondary diagnosis was present at the time of hospital admission may lead to the more accurate identification of preexisting conditions.

OBJECTIVE

To examine the rate of misclassification of ICD-9-CM codes into diagnostic categories by the Dartmouth-Manitoba adaptation of the Charlson index and by the Elixhauser comorbidity algorithm.

DATA SOURCE

Analysis of 178,838 patients in the California State Inpatient Database (CA SID) admitted in 2000 for one of seven major medical and surgical conditions. The CA SID includes a condition present at admission (CPAA) modifier for each ICD-9-CM code.

STUDY DESIGN

The Dartmouth/Charlson index and the Elixhauser comorbidity measure were used to map the ICD-9-CM codes into diagnostic categories for patients in each study population. We calculated the misclassification rate for each mapping algorithm, using information from the CPAA as the "gold standard."

PRINCIPAL FINDINGS

The Dartmouth/Charlson index underestimated the prevalence of hemiplegia/paraplegia by 70 percent, cerebrovascular disease by 70 percent, myocardial infarction by 65 percent, congestive heart failure (CHF) by 45 percent, and peptic ulcer disease by 34 percent. The Elixhauser algorithm misclassified complications as preexisting conditions for 43 percent of the coagulopathies, 25 percent of the fluid and electrolyte disorders, 18 percent of the cardiac arrhythmias, 18 percent of the cardiac arrhythmias, and 9 percent of the cases of CHF.

CONCLUSION

Adding the CPAA modifier to administrative data would significantly enhance the ability of the Dartmouth/Charlson index and of the Elixhauser algorithm to map ICD-9-CM codes to diagnostic categories accurately.

The Relation Between Surgeon Volume and Outcome Following Off-Pump vs On-Pump Coronary Artery Bypass Graft Surgery

STUDY OBJECTIVE

Off-pump coronary artery bypass graft (CABG) surgery has been recently reintroduced into clinical practice. In light of the relatively low level of experience of most cardiac surgeons with off-pump CABG surgery, and the exceptional technical challenge of working on a "beating heart," off-pump CABG surgery presents a unique opportunity to explore the effect of surgeon case volume on surgical outcome after controlling for the effects of patient case mix and hospital volume.

DESIGN

A retrospective cohort study analyzing the association between surgeon volume and in-hospital mortality rate for off-pump and on-pump CABG surgery using random-effects logistic regression modeling.

SETTING AND PATIENTS

The analyses were based on the New York State clinical CABG surgery registry. The study sample consisted of 36,930 patients undergoing isolated CABG surgery between 1998 and 1999 that was performed by 181 surgeons at 33 hospitals.

INTERVENTIONS

None.

RESULTS

There is no association between the number of CABG procedures performed off-pump by an individual surgeon and in-hospital mortality rates (p = 0.93) after controlling for hospital CABG surgery volume and patient-level risk factors. There is also no association between the off-pump CABG surgery mortality rate and the total number of both off-pump and on-pump CABG surgery cases (p = 0.78). In the on-pump CABG surgery cohort, surgeons performing a high volume of CABG procedures had significantly lower risk-adjusted mortality rates among their patients compared to those performing a very low volume, a low-volume, and a medium volume of CABG procedures (p < 0.006).

CONCLUSION

For off-pump CABG surgery, surgeons performing a high volume of procedures do not have better mortality outcomes than those performing a low volume of procedures. However, higher surgeon case volumes are associated with lower mortality rates for on-pump CABG surgery. The absence of a volume-outcome association for off-pump CABG surgery is especially surprising in light of the more technically demanding nature of off-pump CABG surgery compared to on-pump CABG surgery.

Evaluating alternative risk-adjustment strategies for surgery

BACKGROUND

Comparison of institutional health care outcomes requires risk adjustment. Risk-adjustment methodology may influence the results of such comparisons.

METHODS

We compared 3 risk-adjustment methodologies used to assess the quality of surgical care. Nurse reviewers abstracted data from a continuous sample of 2,167 surgical patients at 3 academic institutions. One risk adjustor was based on medical record data (National Surgical Quality Improvement Program [NSQIP]) whereas the other 2, the DxCG and Charlson Comorbidity Index (CCI), primarily used International Classification of Disease-9 (ICD-9) codes. Risk-assessment scores from the 3 systems were compared with each other and with mortality.

RESULTS

Substantial disagreement was found in the risk assessment calculated by the 3 methodologies. Although there was a weak association between the CCI and DxCG, neither correlated well with the NSQIP. The NSQIP was best able to predict mortality, followed by the DxCG and CCI.

CONCLUSION

In surgical patients, different risk-adjustment methodologies afford divergent estimates of mortality risk.

Are diagnosis specific outcome indicators based on administrative data useful in assessing quality of hospital care?

BACKGROUND

Hospital performance reports based on administrative data should distinguish differences in quality of care between hospitals from case mix related variation and random error effects. A study was undertaken to determine which of 12 diagnosis-outcome indicators measured across all hospitals in one state had significant risk adjusted systematic (or special cause) variation (SV) suggesting differences in quality of care. For those that did, we determined whether SV persists within hospital peer groups, whether indicator results correlate at the individual hospital level, and how many adverse outcomes would be avoided if all hospitals achieved indicator values equal to the best performing 20% of hospitals.

METHODS

All patients admitted during a 12 month period to 180 acute care hospitals in Queensland, Australia with heart failure (n = 5745), acute myocardial infarction (AMI) (n = 3427), or stroke (n = 2955) were entered into the study. Outcomes comprised in-hospital deaths, long hospital stays, and 30 day readmissions. Regression models produced standardised, risk adjusted diagnosis specific outcome event ratios for each hospital. Systematic and random variation in ratio distributions for each indicator were then apportioned using hierarchical statistical models.

RESULTS

Only five of 12 (42%) diagnosis-outcome indicators showed significant SV across all hospitals (long stays and same diagnosis readmissions for heart failure; in-hospital deaths and same diagnosis readmissions for AMI; and in-hospital deaths for stroke). Significant SV was only seen for two indicators within hospital peer groups (same diagnosis readmissions for heart failure in tertiary hospitals and inhospital mortality for AMI in community hospitals). Only two pairs of indicators showed significant correlation. If all hospitals emulated the best performers, at least 20% of AMI and stroke deaths, heart failure long stays, and heart failure and AMI readmissions could be avoided.

CONCLUSIONS

Diagnosis-outcome indicators based on administrative data require validation as markers of significant risk adjusted SV. Validated indicators allow quantification of realisable outcome benefits if all hospitals achieved best performer levels. The overall level of quality of care within single institutions cannot be inferred from the results of one or a few indicators.

Licensed nurse staffing and adverse events in hospitals

OBJECTIVE

To examine the changes in licensed nursing staff in Pennsylvania hospitals from 1991 to 1997, and to assess the relationship of licensed nursing staff with patient adverse events in hospitals.

DATA SOURCE

A convenience sample of all Pennsylvania, acute-care, hospitals, 1991 to 1997.

STUDY DESIGN

The study first describes the percentage change of licensed nursing staff categories in Pennsylvania hospitals from 1991 to 1997. Second, random effects Poisson regressions are used to assess the association of the numbers and proportions of licensed nurses with yearly iatrogenic lung collapse, pressure sores, falls, pneumonia, posttreatment infections, and urinary tract infections. Controls are the yearly number of patients, hospital acuity, and other hospital characteristics.

DATA COLLECTION

Secondary data containing patient- and hospital-level measures from three sources were recoded to establish the incidence of adverse events, aggregated to the hospital level, and merged to form one data set. PRINCIPAL FUNDING: Licensed nurses' acuity-adjusted patient load increased from 1991 to 1997. Licensed nurse/total nursing staff declined from 1994 to 1997. Greater incidence of nearly all adverse events occurred in hospitals with fewer licensed nurses. Greater incidence of decubitus ulcers and pneumonia occurred in hospitals with a lower proportion of licensed nurses.

CONCLUSIONS

This study suggests that licensed nurses' patient load began increasing in the 1990s. Adequate licensed nurse staffing is important in minimizing the incidence of adverse events in hospitals. Ensuring adequate licensed nurse staffing should be an area of major concern to hospital management. Improved measures of nurse staffing and patient outcomes, and further studies are suggested.

Mortality trends during a program that publicly reported hospital performance

BACKGROUND

It is unclear whether publicly reporting hospitals' risk-adjusted mortality leads to improvements in outcomes.

OBJECTIVES

To examine mortality trends during a period (1991-1997) when the Cleveland Health Quality Choice program was operational.

RESEARCH DESIGN

Time series.

SUBJECTS

Medicare patients hospitalized with acute myocardial infarction (AMI; n = 10,439), congestive heart failure (CHF; n = 23,505), gastrointestinal hemorrhage (GIH; n = 11,088), chronic obstructive pulmonary disease (COPD; n = 8495), pneumonia (n = 23,719), or stroke (n = 14,293).

MEASURES

Risk-adjusted in-hospital mortality, early postdischarge mortality (between discharge and 30 days after admission), and 30-day mortality.

RESULTS

Risk-adjusted in-hospital mortality declined significantly for all conditions except stroke and GIH, with absolute declines ranging from -2.1% for COPD to -4.8% for pneumonia. However, the mortality rate in the early postdischarge period rose significantly for all conditions except COPD, with increases ranging from 1.4% for GIH to 3.8% for stroke. As a consequence, the 30-day mortality declined significantly only for CHF (absolute decline 1.4%, 95% CI, -2.5 to -0.1%) and COPD (absolute decline 1.6%, 95% CI, -2.8-0.0%). For stroke, risk-adjusted 30-day mortality actually increased by 4.3% (95% CI, 1.8-7.1%).

CONCLUSION

During Cleveland's experiment with hospital report cards, deaths shifted from in hospital to the period immediately after discharge with little or no net reduction in 30-day mortality for most conditions. Hospital profiling remains an unproven strategy for improving outcomes of care for medical conditions. Using in-hospital mortality rates to monitor trends in outcomes for hospitalized patients may lead to spurious conclusions.

Certifying examination performance and patient outcomes following acute myocardial infarction

OBJECTIVE

To establish whether successful certifying examination performances of doctors are associated with their patients' mortality and length of stay following acute myocardial infarction.

DESIGN

Risk adjusted mortality and survivors' length of stay were compared for doctors who had satisfactorily completed training in internal medicine or cardiology and attempted the relevant examination. Specifically, the study investigated the joint effects of hospital location, availability of advanced cardiac care, doctors' specializations, certifying examination performances, year certification was first attempted and patient volume.

SETTING AND PARTICIPANTS

Data on all acute myocardial infarctions in Pennsylvania for the calendar year 1993 were collected by the Pennsylvania Health Care Cost Containment Council. These data were combined with physician information from the database of the American Board of Internal Medicine.

RESULTS

Holding all variables constant, successful examination performance (i.e. certification in internal medicine or cardiology) was associated with a 19% reduction in mortality. Decreased mortality was also correlated with treatment in hospitals located outwith either rural or urban settings and with management by a cardiologist. Shorter stays were not related to examination performance but were associated with treatment by high volume cardiologists who had recently finished training and who cared for their patients in hospitals located outwith rural or urban settings.

CONCLUSIONS

The results of the study add to the evidence supporting the validity of the certifying examination and lend support to the concept that fund of knowledge is related to quality of practice.

The effect of decreasing length of stay on discharge destination and readmission after coronary bypass operation

BACKGROUND

Over the decade of the 1990s, hospital stay after operation declined in response to prospective payment and managed care. As a result, complications previously detected and treated in the hospital may have begun to occur after discharge. In addition, discharge to nursing homes and rehabilitation hospitals may have increased. To address these questions, we used a statewide database to look at the use of postacute care and the 30-day readmission and mortality after coronary bypass operation.

METHODS

A modification of the Commonwealth of Massachusetts Division of Health Care Finance and Policy discharge data to include a unique patient identifier allowed us to retrospectively track patient destination at discharge and study 30-day readmission to all hospitals in the state.

RESULTS

Over the 3-year period after the institution of the unique patient identifier (1993 to 1996), postoperative length of stay after coronary bypass operation decreased from 7.4 to 6 days (19%, P <.0005), but the 30-day readmission rate (17.7%) did not increase. Discharge to rehabilitation hospitals and skilled nursing facilities rose significantly (11.7% to 23.8%), especially in the Medicare population (17.2% to 38.5%). Mortality in the 30 days after discharge remained constant at 0.3%.

CONCLUSIONS

A shorter postoperative length of stay did not appear to disadvantage coronary artery bypass patients by increasing their likelihood of readmission or death. Cost savings from reduced length of stay were offset by increased use of postacute services.

Relationship between coronary angioplasty laboratory volume and outcomes after hospital discharge

BACKGROUND

Although an inverse association has been established between short-term complications of percutaneous coronary interventions (PCIs) and the volume of angioplasty procedures performed by catheterization laboratories, no data are available on the association between laboratory volume and long-term outcomes.

METHODS

A cohort study of 25,222 patients undergoing PCI in 43 laboratories in Pennsylvania from October 1994 to December 1995 was performed by use of the Pennsylvania Health Care Cost Containment Council database. The association of laboratory volume with inhospital, 1-month, and 6-month events was estimated by use of multivariable analyses adjusting for patient and procedural characteristics.

RESULTS

Although a higher volume of procedures was associated with reduced inhospital coronary bypass ([CABG] 0.6 odds ratio [OR] for > or =400 vs <400 PCIs/year; 95% CI 0.4, 0.8), it was not associated with CABG occurring within 1 month after discharge (P =.71; OR 1.0, 95% CI 0.6, 1.7). Laboratory volume was also not significantly associated with postdischarge revascularization (PCI or CABG) at 1 month (P =.58; OR 1.1, 95% CI 0.8, 1.4) or 6 months (P =.47; OR 1.04, 95% CI 0.91, 1.19). In addition, laboratory volume was not associated with rates of myocardial infarction (P =.14), death (P =.28), or the combined outcome of PCI, CABG, myocardial infarction, or death (P =.90) at 1 month after hospital discharge.

CONCLUSIONS

Although our study confirmed the volume/complication relationship for inhospital CABG, it did not reveal an association between volume and postdischarge events. These results suggest that inhospital complications will remain the standard for assessing laboratory volume and that selective use of higher-volume laboratories may not improve long-term outcomes.

Development and validation of a clinical prediction rule for major adverse outcomes in coronary bypass grafting

In this study, we develop and internally validate a clinical prediction rule for in-hospital major adverse outcomes, defined as death, renal failure, reinfarction, cardiac arrest, cerebrovascular accident, or coma, in patients who underwent coronary artery bypass grafting (CABG). All adult patients (n = 9,498) who underwent a CABG and no other concomitant surgery at 12 academic medical centers from August 1993 to October 1995 were included in the study. We assessed in-hospital major adverse outcomes and their predictors using information on admission, coronary angiography, and postoperative hospital course. Predictor variables were limited to information available before the procedure, and outcome variables were represented only by events that occurred postoperatively. We developed and internally validated a clinical prediction rule for any major adverse outcome after CABG. The rule's ability to discriminate outcomes and its calibration were assessed using receiver-operating characteristic analysis and the Hosmer-Lemeshow goodness-of-fit statistic, respectively. A major adverse outcome occurred in 6.5% of patients in the derivation set and 7.2% in the validation set. Death occurred in 2.5% of patients in the derivation set and 2.2% in the validation set. Sixteen variables were independently correlated with major adverse outcomes, with the risk score value attributed to each risk factor ranging from 2 to 12 points. The rule stratified patients into 6 levels of risk based on the total risk score. The spread in probability between the lowest and highest risk groups of having a major adverse outcome was 1.7% to 32.3% in the derivation set and 2.2% to 22.3% in the validation set. The prediction model performed well in both outcome discrimination and calibration. Thus, this clinical prediction rule allows accurate stratification of potential CABG candidates before surgery according to the risk of experiencing a major adverse outcome postoperatively.

Cardiac surgery report cards: comprehensive review and statistical critique

Public report cards and confidential, collaborative peer education represent distinctly different approaches to cardiac surgery quality assessment and improvement. This review discusses the controversies regarding their methodology and relative effectiveness. Report cards have been the more commonly used approach, typically as a result of state legislation. They are based on the presumption that publication of outcomes effectively motivates providers, and that market forces will reward higher quality. Numerous studies have challenged the validity of these hypotheses. Furthermore, although states with report cards have reported significant decreases in risk-adjusted mortality, it is unclear whether this improvement resulted from public disclosure or, rather, from the development of internal quality programs by hospitals. An additional confounding factor is the nationwide decline in heart surgery mortality, including states without quality monitoring. Finally, report cards may engender negative behaviors such as high-risk case avoidance and "gaming" of the reporting system, especially if individual surgeon results are published. The alternative approach, continuous quality improvement, may provide an opportunity to enhance performance and reduce interprovider variability while avoiding the unintended negative consequences of report cards. This collaborative method, which uses exchange visits between programs and determination of best practice, has been highly effective in northern New England and in the Veterans Affairs Administration. However, despite their potential advantages, quality programs based solely on confidential continuous quality improvement do not address the issue of public accountability. For this reason, some states may continue to mandate report cards. In such instances, it is imperative that appropriate statistical techniques and report formats are used, and that professional organizations simultaneously implement continuous quality improvement programs. The statistical methodology underlying current report cards is flawed, and does not justify the degree of accuracy presented to the public. All existing risk-adjustment methods have substantial inherent imprecision, and this is compounded when the results of such patient-level models are aggregated and used inappropriately to assess provider performance. Specific problems include sample size differences, clustering of observations, multiple comparisons, and failure to account for the random component of interprovider variability. We advocate the use of hierarchical or multilevel statistical models to address these concerns, as well as report formats that emphasize the statistical uncertainty of the results.

The effects of quality improvement interventions on inhospital mortality after acute myocardial infarction

OBJECTIVE

To evaluate the effects of quality improvement interventions on inhospital mortality after admission for acute myocardial infarction (AMI).

DESIGN

Before-and-after study (with concurrent controls) based on hospital discharge data from a routinely maintained, administrative database.

SETTING

All Queensland public hospitals, July 1991 - June 1999.

STUDY POPULATION

Patients with AMI admitted through the emergency department.

INTERVENTION

Development and promulgation of clinical practice guidelines at one hospital, combined with regular audit and feedback, commencing November 1995.

MAIN OUTCOME MEASURES

Inhospital mortality (adjusted for age, sex and comorbidities) for four-year periods before (1991-92 to 1994-95) and after (1995-96 to 1998-99) initiation of quality improvement interventions.

RESULTS

Before the intervention, the adjusted odds ratio (OR) for inhospital death at the intervention hospital was about the same as at other public hospitals (adjusted OR, 0.99; 95% CI, 0.80-1.24), but was more than 40% lower after the intervention (adjusted OR, 0.59; 95% Cl, 0.45-0.78). After the intervention, the risk of death at the intervention hospital was lower compared with hospitals with cardiologists as admitting practitioners (adjusted OR, 0.63; 95% CI, 0.48-0.83), with onsite revascularisation facilities (adjusted OR, 0.66; 95% CI, 0.49-0.88), and with large numbers (> or = 250 per year) of annual admissions of patients with AMI (adjusted OR, 0.72; 95% CI, 0.54-0.97).

CONCLUSIONS

Quality improvement interventions lower the risk of inhospital death in patients with AMI. Implementation of such interventions in all hospitals may confer a risk of death lower than that achieved by admitting all patients under the care of cardiologists, or to hospitals with revascularisation facilities or a high volume of admissions of patients with AMI.

A simple benchmark for evaluating quality of care of patients following acute myocardial infarction

OBJECTIVE

To develop a simple risk model as a basis for evaluating care of patients admitted with acute myocardial infarction.

METHODS

From coronary care registers, biochemistry records and hospital management systems, 2153 consecutive patients with confirmed acute myocardial infarction were identified. With 30 day all cause mortality as the end point, a multivariable logistic regression model of risk was constructed and validated in independent patient cohorts. The areas under receiver operating characteristic curves were calculated as an assessment of sensitivity and specificity. The model was reapplied to a number of commonly studied subgroups for further assessment of robustness.

RESULTS

A three variable model was developed based on age, heart rate, and systolic blood pressure on admission. This produced an individual probability of death by 30 days (P30) where P30 = 1/(1 + exp(−L30)) and L30 = −5.624 + (0.085 × age) + (0.014 × heart rate) − (0.022 × systolic blood pressure). The areas under the receiver operating characteristic curves for the reference and test cohorts were 0.79 (95% CI 0.76 to 0.82) and 0.76 (95% CI 0.72 to 0.79), respectively. To aid application of the model to routine clinical audit, a normogram relating observed mortality and sample size to the likelihood of a significant deviation from the expected 30 day mortality rate was constructed.

CONCLUSIONS

This risk model is simple, reproducible, and permits quality of care of acute myocardial infarction patients to be reliably evaluated both within and between centres.

A simple benchmark for evaluating quality of care of patients following acute myocardial infarction

OBJECTIVE

To develop a simple risk model as a basis for evaluating care of patients admitted with acute myocardial infarction.

METHODS

From coronary care registers, biochemistry records and hospital management systems, 2153 consecutive patients with confirmed acute myocardial infarction were identified. With 30 day all cause mortality as the end point, a multivariable logistic regression model of risk was constructed and validated in independent patient cohorts. The areas under receiver operating characteristic curves were calculated as an assessment of sensitivity and specificity. The model was reapplied to a number of commonly studied subgroups for further assessment of robustness.

RESULTS

A three variable model was developed based on age, heart rate, and systolic blood pressure on admission. This produced an individual probability of death by 30 days (P(30)) where P(30) = 1/(1 + exp(-L(30))) and L(30) = -5.624 + (0.085 x age) + (0.014 x heart rate) - (0.022 x systolic blood pressure). The areas under the receiver operating characteristic curves for the reference and test cohorts were 0.79 (95% CI 0.76 to 0.82) and 0.76 (95% CI 0.72 to 0.79), respectively. To aid application of the model to routine clinical audit, a normogram relating observed mortality and sample size to the likelihood of a significant deviation from the expected 30 day mortality rate was constructed.

CONCLUSIONS

This risk model is simple, reproducible, and permits quality of care of acute myocardial infarction patients to be reliably evaluated both within and between centres.

Development and validation of the Ontario acute myocardial infarction mortality prediction rules

OBJECTIVES

To develop and validate simple statistical models that can be used with hospital discharge administrative databases to predict 30-day and one-year mortality after an acute myocardial infarction (AMI).

BACKGROUND

There is increasing interest in developing AMI "report cards" using population-based hospital discharge databases. However, there is a lack of simple statistical models that can be used to adjust for regional and interinstitutional differences in patient case-mix.

METHODS

We used linked administrative databases on 52,616 patients having an AMI in Ontario, Canada, between 1994 and 1997 to develop logistic regression statistical models to predict 30-day and one-year mortality after an AMI. These models were subsequently validated in two external cohorts of AMI patients derived from administrative datasets from Manitoba, Canada, and California, U.S.

RESULTS

The 11-variable Ontario AMI mortality prediction rules accurately predicted mortality with an area under the receiver operating characteristic (ROC) curve of 0.78 for 30-day mortality and 0.79 for one-year mortality in the Ontario dataset from which they were derived. In an independent validation dataset of 4,836 AMI patients from Manitoba, the ROC areas were 0.77 and 0.78, respectively. In a second validation dataset of 112,234 AMI patients from California, the ROC areas were 0.77 and 0.78 respectively.

CONCLUSIONS

The Ontario AMI mortality prediction rules predict quite accurately 30-day and one-year mortality after an AMI in linked hospital discharge databases of AMI patients from Ontario, Manitoba and California. These models may also be useful to outcomes and quality measurement researchers in other jurisdictions.

Monitoring health reform: a report card approach

During the past several years, budget cuts have forced hospitals in several countries to change the way they deliver care. Gilson (Gilson, L. (1998).

DISCUSSION

In defence and pursuit of equity. Social Science & Medicine, 47(12), 1891-1896) has argued that, while health reforms are designed to improve efficiency, they have considerable potential to harm equity in the delivery of health care services. It is essential to monitor the impact of health reforms, not only to ensure the balance between equity and efficiency, but also to determine the effect of reforms on such things as access to care and the quality of care delivered. This paper proposes a framework for monitoring these and other indicators that may be affected by health care reform. Application of this framework is illustrated with data from Winnipeg, Manitoba, Canada. Despite the closure of almost 24% of the hospital beds in Winnipeg between 1992 and 1996, access to care and quality of care remained generally unchanged. Improvements in efficiency occurred without harming the equitable delivery of health care services. Given our increasing understanding of the weak links between health care and health, improving efficiency within the health care system may actually be a prerequisite for addressing equity issues in health.

Certification and specialization: do they matter in the outcome of acute myocardial infarction?

PURPOSE

To learn whether there are differences among certified and self-designated cardiologists, internists, and family practitioners in terms of the mortality of their patients with acute myocardial infarction (AMI).

METHOD

Data on all patients admitted with AMI were collected for calendar year 1993 by the Pennsylvania Health Care Cost Containment Council and analyzed. Certified and self-designated family practitioners, internists, and cardiologists (n = 4,546) were compared with respect to the characteristics of their patients' illnesses. In addition, a regression model was fitted in which mortality was the dependent measure and the independent variables were the probability of death, hospital characteristics (location and the availability of advanced cardiac care), and physician characteristics (patient volume, years since graduation from medical school, specialty, and certification status).

RESULTS

On average, cardiologists treated more patients than did generalists, and their patients were less severely ill. In the regression analysis, all variables were statistically significant except the availability of advanced cardiac care. Holding all other variables constant, treatment by a certified physician was associated with a 15% reduction in mortality among patients with AMI.

CONCLUSIONS

Less patient mortality was associated with treatment by physicians who were cardiologists, cared for larger numbers of AMI patients, were closer to their graduation from medical school, and were certified.

Does clinical evidence support ICD-9-CM diagnosis coding of complications?

BACKGROUND

Hospital discharge diagnoses, coded by use of the International Classification of Diseases, 9th Revision, Clinical Modification (ICD-9-CM), increasingly determine reimbursement and support quality monitoring. Prior studies of coding validity have investigated whether coding guidelines were met, not whether the clinical condition was actually present.

OBJECTIVE

To determine whether clinical evidence in medical records confirms selected ICD-9-CM discharge diagnoses coded by hospitals.

RESEARCH DESIGN AND SUBJECTS

Retrospective record review of 485 randomly sampled 1994 hospitalizations of elderly Medicare beneficiaries in Califomia and Connecticut.

MAIN OUTCOME MEASURE

Proportion of patients with specified ICD-9-CM codes representing potential complications who had clinical evidence confirming the coded condition.

RESULTS

Clinical evidence supported most postoperative acute myocardial infarction diagnoses, but fewer than 60% of other diagnoses had confirmatory clinical evidence by explicit clinical criteria; 30% of medical and 19% of surgical patients lacked objective confirmatory evidence in the medical record. Across 11 surgical and 2 medical complications, objective clinical criteria or physicians' notes supported the coded diagnosis in >90% of patients for 2 complications, 80% to 90% of patients for 4 complications, 70% to <80% of patients for 5 complications, and <70% for 2 complications. For some complications (postoperative pneumonia, aspiration pneumonia, and hemorrhage or hematoma), a large fraction of patients had only a physician's note reporting the complication.

CONCLUSIONS

Our findings raise questions about whether the clinical conditions represented by ICD-9-CM codes used by the Complications Screening Program were in fact always present. These findings highlight concerns about the clinical validity of using ICD-9-CM codes for quality monitoring.

Use of administrative data to find substandard care: validation of the complications screening program

OBJECTIVE

The use of administrative data to identify inpatient complications is technically feasible and inexpensive but unproven as a quality measure. Our objective was to validate whether a screening method that uses data from standard hospital discharge abstracts identifies complications of care and potential quality problems.

DESIGN

This was a case-control study with structured implicit physician reviews.

SETTING

Acute-care hospitals in California and Connecticut in 1994.

PATIENTS

The study included 1,025 Medicare beneficiaries greater than 265 years of age.

METHODS

Using administrative data, we stratified acute-care hospitals by observed-to-expected complication rates and randomly selected hospitals within each state. We randomly selected cases flagged with 1 of 17 surgical complications and 6 medical complications. We randomly selected controls from unflagged cases.

MAIN OUTCOME MEASURE

Peer-review organization physicians' judgments about the presence of the flagged complication and potential quality-of-care problems.

RESULTS

Physicians confirmed flagged complications in 68.4% of surgical and 27.2% of medical cases. They identified potential quality problems in 29.5% of flagged surgical and 15.7% of medical cases but in only 2.1% of surgical and medical controls. The rate of physician-identified potential quality problems among flagged cases exceeded 25% in 9 surgical screens and 1 medical screen. Reviewers noted several potentially mitigating circumstances that affected their judgments about quality, including factors related to the patients' illness, the complexity of the case, and technical difficulties that clinicians encountered.

CONCLUSIONS

For some types of complications, screening administrative data may offer an efficient approach for identifying potentially problematic cases for physician review. Understanding the basis for physicians' judgments about quality requires more investigation.

Results of report cards for patients with congestive heart failure depend on the method used to adjust for severity

BACKGROUND

The validity of outcome report cards may depend on the ways in which they are adjusted for risk.

OBJECTIVES

To compare the predictive ability of generic and disease-specific survival prediction models appropriate for use in patients with heart failure, to simulate outcome report cards by comparing survival across hospitals and adjusting for severity of illness using these models, and to assess the ways in which the results of these comparisons depend on the adjustment method.

DESIGN

Analysis of data from a prospective cohort study.

SETTING

A university hospital, a Veterans Affairs (VA) medical center, and a community hospital.

PATIENTS

Sequential patients presenting in the emergency department with acute congestive heart failure.

MEASUREMENTS

Unadjusted 30-day and 1-year mortality across hospitals and 30-day and 1-year mortality adjusted by using disease-specific survival prediction models (two sickness-at-admission models, the Cleveland Health Quality Choice model, the Congestive Heart Failure Mortality Time-Independent Predictive Instrument) and generic models (Acute Physiology and Chronic Health Evaluation [APACHE] II, APACHE III, the mortality prediction model, and the Chadson comorbidity index).

RESULTS

The community hospital's unadjusted 30-day survival rate (85.0%) and the VA medical center's unadjusted 1-year survival rate (60.9%) were significantly lower than corresponding rates at the university hospital (92.7% and 67.5%, respectively). No severity model had excellent ability to discriminate patients by survival rates (all areas under the receiver-operating characteristic curve < 0.73). Whether the VA medical center, the community hospital, both, or neither had worse survival rates on simulated report cards than the university hospital depended on the prediction model used for adjustment.

CONCLUSIONS

Results of simulated outcome report cards for survival in patients with congestive heart failure depend on the method used to adjust for severity.

The association between hospital volume and survival after acute myocardial infarction in elderly patients

BACKGROUND

Patients with chest pain thought to be due to acute coronary ischemia are typically taken by ambulance to the nearest hospital. The potential benefit of field triage directly to a hospital that treats a large number of patients with myocardial infarction is unknown.

METHODS

We conducted a retrospective cohort study of the relation between the number of Medicare patients with myocardial infarction that each hospital in the study treated (hospital volume) and long-term survival among 98,898 Medicare patients 65 years of age or older. We used proportional-hazards methods to adjust for clinical, demographic, and health-system-related variables, including the availability of invasive procedures, the specialty of the attending physician, and the area of residence of the patient (rural, urban, or metropolitan).

RESULTS

The patients in the quartile admitted to hospitals with the lowest volume were 17 percent more likely to die within 30 days after admission than patients in the quartile admitted to hospitals with the highest volume (hazard ratio, 1.17; 95 percent confidence interval, 1.09 to 1.26; P<0.001), which resulted in 2.3 more deaths per 100 patients. The crude mortality rate at one year was 29.8 percent among the patients admitted to the lowest-volume hospitals, as compared with 27.0 percent among those admitted to the highest-volume hospitals. There was a continuous inverse dose-response relation between hospital volume and the risk of death. In an analysis of subgroups defined according to age, history of cardiac disease, Killip class of infarction, presence or absence of contraindications to thrombolytic therapy, and time from the onset of symptoms, survival at high-volume hospitals was consistently better than at low-volume hospitals. The availability of technology for angioplasty and bypass surgery was not independently associated with overall mortality.

CONCLUSIONS

Patients with acute myocardial infarction who are admitted directly to hospitals that have more experience treating myocardial infarction, as reflected by their case volume, are more likely to survive than are patients admitted to low-volume hospitals.

Screening inpatient quality using post-discharge events

BACKGROUND

Decreasing hospital lengths of stay (LOS) hamper efforts to detect and to definitively treat complications of care. Patients leave before some complications are identified.

OBJECTIVES

To develop a computerized method to screen for hospital complications using readily available administrative data from outpatient and nonacute care within 90 days of discharge.

DESIGN

We developed the Complications Screening Program for Outpatient data (CSP-O) by using diagnosis and procedure codes from Medicare Part A and B claims to define 50 complication screens. Seventeen apply to specific procedural cases, and 33 apply to all adult, acute, medical, or surgical hospitalizations. The CSP-O algorithm examined outpatient, physician office, home health agency, and hospice claims within 90 days following discharge.

SUBJECTS

Seven hundred thirty nine thousand, two hundred and forty eight discharges of Medicare beneficiaries (age range, > or = 65 years) were admitted to 515 hospitals nationwide in 1994.

RESULTS

Complete 90-day, post-discharge windows were present for 62.8% of all and 68.5% of procedural cases. The 33 general screens flagged 13.6% of all cases; only 1.8% of procedural cases were flagged by the 17 procedural screens. When we allowed the CSP-O algorithm to scan information from acute hospital readmissions, flag rates rose to 32.8% for general and 8.7% for procedural complications. Controlling for patient and hospital characteristics, flag rates were considerably higher among the very old and at small and for-profit institutions.

CONCLUSIONS

Whereas several CSP-O findings have construct validity, limitations of claims raise concerns. Regardless of the CSPO's ultimate utility, examining post-discharge experiences to identify inpatient complications remains important as LOSs fall.

Promises and perils of managed care for older patients with cardiac disease

Managed care for elderly patients with cardiovascular disease will likely become more widespread because it has the propensity to provide comprehensive care while controlling costs. The issues of comorbidity and the changes of aging per se become increasingly important with increasing age and emphasize the need to use the basic principles of clinical decision making in the management of elderly patients, rather than focusing exclusively on a specific acute problem. Currently, there is great turmoil in the managed care market for the elderly with multiple models being tried with highly variable degrees of success from both the financial and patient satisfaction perspective. Physician leadership can be a critical element in resolving many of the dilemmas.