Changing analytical approaches in European epidemiology -- a short comment on a recent article

Disentangling the contribution of hospitals and municipalities for understanding patient level differences in one-year mortality risk after hip-fracture: A cross-classified multilevel analysis in Sweden

Background

One-year mortality after hip-fracture is a widely used outcome measure when comparing hospital care performance. However, traditional analyses do not explicitly consider the referral of patients to municipality care after just a few days of hospitalization. Furthermore, traditional analyses investigates hospital (or municipality) variation in patient outcomes in isolation rather than as a component of the underlying patient variation. We therefore aimed to extend the traditional approach to simultaneously estimate both case-mix adjusted hospital and municipality comparisons in order to disentangle the amount of the total patient variation in clinical outcomes that was attributable to the hospital and municipality level, respectively.

Methods

We determined 1-year mortality risk in patients aged 65 or above with hip fractures registered in Sweden between 2011 and 2014. We performed cross-classified multilevel analysis with 54,999 patients nested within 54 hospitals and 290 municipalities. We adjusted for individual demographic, socioeconomic and clinical characteristics. To quantify the size of the hospital and municipality variation we calculated the variance partition coefficient (VPC) and the area under the receiver operator characteristic curve (AUC).

Results

The overall 1-year mortality rate was 25.1%. The case-mix adjusted rates varied from 21.7% to 26.5% for the 54 hospitals, and from 18.9% to 29.5% for the 290 municipalities. The VPC was just 0.2% for the hospital and just 0.1% for the municipality level. Patient sociodemographic and clinical characteristics were strong predictors of 1-year mortality (AUC = 0.716), but adding the hospital and municipality levels in the cross-classified model had a minor influence (AUC = 0.718).

Conclusions

Overall in Sweden, one-year mortality after hip-fracture is rather high. However, only a minor part of the patient variation is explained by the hospital and municipality levels. Therefore, a possible intervention should be nation-wide rather than directed to specific hospitals or municipalities.

Albuminuria measurement in diabetic care: a multilevel analysis measuring the influence of accreditation on institutional performance

Background

Studies assessing institutional performance regarding quality of care are frequently performed using single-level statistical analyses investigating differences between provider averages of various quality indicators. However, such analyses are insufficient as they do not consider patients’ heterogeneity around those averages. Hence, we apply a multilevel analysis of individual-patient heterogeneity that distinguishes between ‘general’ (‘latent quality’ or measures of variance) and ‘specific’ (measures of association) contextual effects. We assess general contextual effects of the hospital departments and the specific contextual effect of a national accreditation programme on adherence to the standard benchmark for albuminuria measurement in Danish patients with diabetes.

Methods

From the Danish Adult Diabetes Database, we extracted data on 137 893 patient cases admitted to hospitals between 2010 and 2013. Applying multilevel logistic and probit regression models for every year, we quantified general contextual effects of hospital department by the intraclass correlation coefficient (ICC) and the area under the receiver operating characteristic curve (AUC) values. We evaluated the specific effect of hospital accreditation using the ORs and the change in the department variance.

Results

In 2010, the department context had considerable influence on adherence with albuminuria measurement (ICC=21.8%, AUC=0.770), but the general effect attenuated along with the implementation of the national accreditation programme. The ICC value was 16.5% in 2013 and the rate of compliance with albuminuria measurement increased from 91.6% in 2010 to 96% in 2013.

Conclusions

Parallel to implementation of the national accreditation programme, departments’ compliance with the standard benchmark for albuminuria measurement increased and the ICC values decreased, but remained high. While those results indicate an overall quality improvement, further intervention focusing on departments with the lowest compliance could be considered.

Short Term Survival after Admission for Heart Failure in Sweden: Applying Multilevel Analyses of Discriminatory Accuracy to Evaluate Institutional Performance

BACKGROUND

Hospital performance is frequently evaluated by analyzing differences between hospital averages in some quality indicators. The results are often expressed as quality charts of hospital variance (e.g., league tables, funnel plots). However, those analyses seldom consider patients heterogeneity around averages, which is of fundamental relevance for a correct evaluation. Therefore, we apply an innovative methodology based on measures of components of variance and discriminatory accuracy to analyze 30-day mortality after hospital discharge with a diagnosis of Heart Failure (HF) in Sweden.

METHODS

We analyzed 36,943 patients aged 45-80 treated in 565 wards at 71 hospitals during 2007-2009. We applied single and multilevel logistic regression analyses to calculate the odds ratios and the area under the receiver-operating characteristic (AUC). We evaluated general hospital and ward effects by quantifying the intra-class correlation coefficient (ICC) and the increment in the AUC obtained by adding random effects in a multilevel regression analysis (MLRA). Finally, the Odds Ratios (ORs) for specific ward and hospital characteristics were interpreted jointly with the proportional change in variance (PCV) and the proportion of ORs in the opposite direction (POOR).

FINDINGS

Overall, the average 30-day mortality was 9%. Using only patient information on age and previous hospitalizations for different diseases we obtained an AUC = 0.727. This value was almost unchanged when adding sex, country of birth as well as hospitals and wards levels. Average mortality was higher in small wards and municipal hospitals but the POOR values were 15% and 16% respectively.

CONCLUSIONS

Swedish wards and hospitals in general performed homogeneously well, resulting in a low 30-day mortality rate after HF. In our study, knowledge on a patient's previous hospitalizations was the best predictor of 30-day mortality, and this information did not improve by knowing the sex and country of birth of the patient or where the patient was treated.

Inter-doctor variations in the assessment of functional incapacities by insurance physicians

Background

The aim of this study was to determine the - largely unexplored - extent of systematic variation in the work disability assessment by Dutch insurance physicians (IPs) of employees on long-term sick leave, and to ascertain whether this variation was associated with the individual characteristics and opinions of IPs.

Methods

In March 2008 we conducted a survey among IPs on the basis of the 'Attitude - Social norm - self-Efficacy' (ASE) model. We used the ensuing data to form latent variables for the ASE constructs. We then linked the background variables and the measured constructs for IPs (n = 199) working at regional offices (n = 27) to the work disability assessments of clients (n = 83,755) and their characteristics. These assessments were carried out between July 2003 and April 2008. We performed multilevel regression analysis on three important assessment outcomes: No Sustainable Capacity or Restrictions for Working Hours (binominal), Functional Incapacity Score (scale 0-6) and Maximum Work Disability Class (binominal). We calculated Intra Class Correlations (ICCs) at IP level and office level and explained variances (R²) for the three outcomes. A higher ICC reflects stronger systematic variation.

Results

The ICCs at IP level were approximately 6% for No Sustainable Capacity or Restrictions for Working Hours and Maximum Work Disability Class and 12% for Functional Incapacity Score. Background IP variables and the measured ASE constructs for physicians contributed very little to the variation - at most 1%. The ICCs at office level ranged from 0% to around 1%. The R² was 11% for No Sustainable Capacity or Restrictions for Working Hours, 19% for Functional Incapacity Score and 37% for Maximum Work Disability Class.

Conclusion

Our study uncovered small to moderate systematic variations in the outcome of disability assessments in the Netherlands. However, the individual characteristics and opinions of insurance physicians have very little impact on these variations. Our findings provided no indications of other reasons for these variations. They may be related to different work routines or to different views on the workload of a 'normal' employee. If so, they could be reduced by well-developed and comprehensively implemented guidelines. Therefore, further research is needed.

Differences between family practices in the associations of patient characteristics with health care experiences

When comparing health care providers, patient experience data are usually adjusted for case-mix associations to ensure fair comparisons. Previous studies in the United States showed that case-mix associations sometimes vary across health care providers. Such variation could indicate differential provider behavior for patient subgroups, in which case current adjustment techniques might be inappropriate. To see whether this variation is also apparent in a health care system different from the U.S. system, the authors analyzed Dutch patients' experiences with family practice care. Using multilevel random slope models, the associations between age, general health status, mental health status, education, sex, and ethnicity on one hand and reported experiences on the other hand were assessed across family practices. The authors found only five significant variances between case-mix coefficients, all for outcomes related to health care professionals' conduct. These findings correspond to previous U.S. findings, suggesting that the case-mix variations reported here and previously constitute a rather robust phenomenon.

Multilevel survival analysis of health inequalities in life expectancy

BACKGROUND

The health status of individuals is determined by multiple factors operating at both micro and macro levels and the interactive effects of them. Measures of health inequalities should reflect such determinants explicitly through sources of levels and combining mean differences at group levels and the variation of individuals, for the benefits of decision making and intervention planning. Measures derived recently from marginal models such as beta-binomial and frailty survival, address this issue to some extent, but are limited in handling data with complex structures. Beta-binomial models were also limited in relation to measuring inequalities of life expectancy (LE) directly.

METHODS

We propose a multilevel survival model analysis that estimates life expectancy based on survival time with censored data. The model explicitly disentangles total health inequalities in terms of variance components of life expectancy compared to the source of variation at the level of individuals in households and parishes and so on, and estimates group differences of inequalities at the same time. Adjusted distributions of life expectancy by gender and by household socioeconomic level are calculated. Relative and absolute health inequality indices are derived based on model estimates. The model based analysis is illustrated on a large Swedish cohort of 22,680 men and 26,474 women aged 6569 in 1970 and followed up for 30 years. Model based inequality measures are compared to the conventional calculations.

RESULTS

Much variation of life expectancy is observed at individual and household levels. Contextual effects at Parish and Municipality level are negligible. Women have longer life expectancy than men and lower inequality. There is marked inequality by the level of household socioeconomic status measured by the median life expectancy in each socio-economic group and the variation in life expectancy within each group.

CONCLUSION

Multilevel survival models are flexible and efficient tools in studying health inequalities of life expectancy or survival time data with a geographic structure of more than 2 levels. They are complementary to conventional methods and override some limitations of marginal models. Future research on determinants of health inequalities in the LE of the specific cohort on the household and individual factors could reveal some important causes over the marked household level inequalities.

The choice between different statistical approaches to risk-adjustment influenced the identification of outliers

OBJECTIVE

Many statistical approaches have been applied to compare health care providers' performance, but few studies have examined how the outlier status of providers depends on the choice between risk-adjustment techniques.

STUDY DESIGN AND SETTING

We analyzed the recourse to breast-conserving surgery (BCS) for breast carcinoma across 31 hospitals of the Veneto Region (Italy). The following methods were compared: the ratio of observed to expected events (O/E), regression models with provider effects introduced as dummy variables obtained by standard or weighted effect coding, and multilevel analysis.

RESULTS

The O/E method classified seven hospitals (one with high and six with low BCS rates) as outliers. The regression model with the weighted parameterization gave similar results, whereas through standard effect coding all odds ratios shifted and different outliers were identified. Multilevel analysis was quite conservative in identifying small hospitals with BCS rates lower than the regional mean.

CONCLUSION

Whenever feasible, results obtained through different statistical methodologies should be compared. If providers are modeled as dummy variables obtained by effect coding, departures of the model intercept from the regional mean should be checked. The increasing use of multilevel models could entail a lower sensitivity in identifying low-quality outliers if a volume-outcome relationship exists.