Measures of discriminatory accuracy and categorizations in public health: a response to Allan Krasnik's editorial

Multilevel modelling for measuring interaction of effects between multiple categorical variables: An illustrative application using risk factors for preeclampsia

BACKGROUND

Measuring multiple and higher-order interaction effects between multiple categorical variables proves challenging.

OBJECTIVES

To illustrate a multilevel modelling approach to studying complex interactions.

METHODS

We apply a two-level random-intercept linear regression to a binary outcome for individuals (level-1) nested within strata (level-2) defined by all observed combinations of multiple categorical exposure variables. As a pedagogic application, we analyse 36 strata defined by five risk factors of preeclampsia (parity, previous preeclampsia, chronic hypertension, multiple pregnancies, body mass index category) among 652,603 women in the Swedish Medical Birth Registry between 2002 and 2010.

RESULTS

The absolute risk of preeclampsia was 4% but was predicted to vary from 1% to 44% across strata. The stratum discriminatory accuracy was 30% according to the variance partition coefficient (VPC) and 0.73 according to the area under the receiver operating characteristic curve (AUC). While the risk heterogeneity across strata was primarily due to the main effects of the categories defining the strata, 5% of the variation was attributable to their two- and higher-way interaction effects. One stratum presented a positive interaction, and two strata presented negative interaction.

CONCLUSIONS

Multilevel modelling is an innovative tool for identifying and analysing higher-order interaction effects. Further work is needed to explore how this approach can best be applied to making causal inferences.

Precision public health: Mapping socioeconomic disparities in opioid dispensations at Swedish pharmacies by Multilevel Analysis of Individual Heterogeneity and Discriminatory Accuracy (MAIHDA)

BACKGROUND

In light of the opioid epidemic in the United States, there is growing concern about the use of opioids in Sweden as it may lead to misuse and overuse and, in turn, severe public health problems. However, little is known about the distribution of opioid use across different demographic and socioeconomic dimensions in the Swedish general population. Therefore, we applied an intersectional Multilevel Analysis of Individual Heterogeneity and Discriminatory Accuracy (MAIHDA), to obtain an improved mapping of the risk heterogeneity of and socioeconomic inequalities in opioid prescription receipt.

METHODS AND FINDINGS

Using data from 6,846,106 residents in Sweden aged 18 and above, we constructed 72 intersectional strata from combinations of gender, age, income, cohabitation status, and presence or absence of psychological distress. We modelled the absolute risk (AR) of opioid prescription receipt in a series of multilevel logistic regression models distinguishing between additive and interaction effects. By means of the Variance Partitioning Coefficient (VPC) and the area under the receiver operating characteristic curve (AUC), we quantified the discriminatory accuracy (DA) of the intersectional strata for discerning those who received opioid prescriptions from those who did not. The AR of opioid prescription receipt ranged from 2.77% (95% CI 2.69-2.86) among low-income men aged 18-34, living alone, without psychological distress, to 28.25% (95% CI 27.95-28.56) among medium-income women aged 65 and older, living alone, with psychological distress. In a model that conflated both additive and interaction effects, the intersectional strata had a fair DA for discerning opioid users from non-users (VPC = 13.2%, AUC = 0.68). However, in the model that decomposed total effects into additive and interaction effects, the VPC was very low (0.42%) indicating the existence of small interaction effects for a number of the intersectional strata.

CONCLUSIONS

The intersectional MAIHDA approach aligns with the aims of precision public health, through improving the evidence base for health policy by increasing understanding of both health inequalities and individual heterogeneity. This approach is particularly relevant for socioeconomically conditioned outcomes such as opioid prescription receipt. We have identified intersections of social position within the Swedish population at greater risk for opioid prescription receipt.

Factors associated with 30-day readmission for patients hospitalized for seizures

Background

We sought to determine the cumulative incidence of readmissions after a seizure-related hospitalization and identify risk factors and readmission diagnoses.

Methods

We performed a retrospective cohort study of adult patients hospitalized with a primary discharge diagnosis of seizure (International Classification of Diseases, Ninth Edition, Clinical Modification codes 345.xx and 780.3x) using the State Inpatient Databases across 11 states from 2009 to 2012. Hospital and community characteristics were obtained from the American Hospital Association and Robert Wood Johnson Foundation. We performed logistic regressions to explore effects of patient, hospital, and community factors on readmissions within 30 days of discharge.

Results

Of 98,712 patients, 13,929 (14%) were readmitted within 30 days. Reasons for readmission included epilepsy/convulsions (30% of readmitted patients), mood disorders (5%), schizophrenia (4%), and septicemia (4%). The strongest predictors of readmission were diagnoses of CNS tumor (odds ratio [OR] 2.1, 95% confidence interval [CI] 1.9-2.4) or psychosis (OR 1.8, 95% CI 1.7-1.8), urgent index admission (OR 2.0, 95% CI 1.8-2.2), transfer to nonacute facilities (OR 1.7, 95% CI 1.6-1.8), long length of stay (OR 1.7, 95% CI 1.6-1.8), and for-profit hospitals (OR 1.7, 95% CI 1.6-1.8). Our main model's c-statistic was 0.66. Predictors of readmission for status epilepticus included index admission for status epilepticus (OR 3.5, 95% CI 2.6-4.7), low hospital epilepsy volume (OR 0.4, 95% CI 0.3-0.7), and rural hospitals (OR 4.8, 95% CI 2.1-10.9).

Conclusion

Readmission is common after hospitalization for seizures. Prevention strategies should focus on recurrent seizures, the most common readmission diagnosis. Many factors were associated with readmission, although readmissions remain challenging to predict.

Socioeconomic differences in body mass index in Spain: An intersectional multilevel analysis of individual heterogeneity and discriminatory accuracy

Many studies have demonstrated the existence of simple, unidimensional socioeconomic gradients in body mass index (BMI). However, in the present paper we move beyond such traditional analyses by simultaneously considering multiple demographic and socioeconomic dimensions. Using the Spanish National Health Survey 2011–2012, we apply intersectionality theory and multilevel analysis of individual heterogeneity and discriminatory accuracy (MAIHDA) to analyze 14,190 adults nested within 108 intersectional strata defined by combining categories of gender, age, income, educational achievement and living situation. We develop two multilevel models to obtain information on stratum-specific BMI averages and the degree of clustering of BMI within strata expressed by the intra-class correlation coefficient (ICC). The first model is a simple variance components analysis that provides a detailed mapping of the BMI disparities in the population and measures the accuracy of stratum membership to predict individual BMI. The second model includes the variables used to define the intersectional strata as a way to identify stratum-specific interactions. The first model suggests moderate but meaningful clustering of individual BMI within the intersectional strata (ICC = 12.4%). Compared with the population average (BMI = 26.07 Kg/m2), the stratum of cohabiting 18-35-year-old females with medium income and high education presents the lowest BMI (-3.7 Kg/m2), while cohabiting 36-64-year-old females with low income and low education show the highest BMI (+2.6 Kg/m2). In the second model, the ICC falls to 1.9%, suggesting the existence of only very small stratum specific interaction effects. We confirm the existence of a socioeconomic gradient in BMI. Compared with traditional analyses, the intersectional MAIHDA approach provides a better mapping of socioeconomic and demographic inequalities in BMI. Because of the moderate clustering, public health policies aiming to reduce BMI in Spain should not solely focus on the intersectional strata with the highest BMI, but should also consider whole population polices.

Chronic Obstructive Pulmonary Disease in Sweden: An intersectional multilevel analysis of individual heterogeneity and discriminatory accuracy

Socioeconomic, ethnic and gender disparities in Chronic Obstructive Pulmonary Disease (COPD) risk are well established but no studies have applied multilevel analysis of individual heterogeneity and discriminatory accuracy (MAIHDA) within an intersectional framework to study this outcome. We study individuals at the first level of analysis and combinations of multiple social and demographic categorizations (i.e., intersectional strata) at the second level of analysis. Here we used MAIHDA to assess to what extent individual differences in the propensity of developing COPD are at the intersectional strata level. We also used MAIHDA to determine the degree of similarity in COPD incidence of individuals in the same intersectional stratum. This leads to an improved understanding of risk heterogeneity and of the social dynamics driving socioeconomic and demographic disparities in COPD incidence. Using data from 2,445,501 residents in Sweden aged 45–65, we constructed 96 intersectional strata combining categories of age, gender, income, education, civil- and migration status. The incidences of COPD ranged from 0.02% for young, native males with high income and high education who cohabited to 0.98% for older native females with low income and low education who lived alone. We calculated the intra-class correlation coefficient (ICC) that informs on the discriminatory accuracy of the categorizations. In a model that conflated additive and interaction effects, the ICC was good (20.0%). In contrast, in a model that measured only interaction effects, the ICC was poor (1.1%) suggesting that most of the observed differences in COPD incidence across strata are due to the main effects of the categories used to construct the intersectional matrix while only a minor share of the differences are attributable to intersectional interactions. We found conclusive interaction effects. The intersectional MAIHDA approach offers improved information to guide public health policies in COPD prevention, and such policies should adopt an intersectional perspective.

•

We use multilevel analysis of individual heterogeneity and discriminatory accuracy.

•

There is a clear difference in COPD incidence between intersectional strata.

•

Intersectionality improves mapping of socioeconomic differences in COPD incidence.

•

Preventive measures should be based on intersectional rather than classic analyses.

Predictive modeling of emergency cesarean delivery

OBJECTIVE

To increase discriminatory accuracy (DA) for emergency cesarean sections (ECSs).

STUDY DESIGN

We prospectively collected data on and studied all 6,157 births occurring in 2014 at four public hospitals located in three different autonomous communities of Spain. To identify risk factors (RFs) for ECS, we used likelihood ratios and logistic regression, fitted a classification tree (CTREE), and analyzed a random forest model (RFM). We used the areas under the receiver-operating-characteristic (ROC) curves (AUCs) to assess their DA.

RESULTS

The magnitude of the LR+ for all putative individual RFs and ORs in the logistic regression models was low to moderate. Except for parity, all putative RFs were positively associated with ECS, including hospital fixed-effects and night-shift delivery. The DA of all logistic models ranged from 0.74 to 0.81. The most relevant RFs (pH, induction, and previous C-section) in the CTREEs showed the highest ORs in the logistic models. The DA of the RFM and its most relevant interaction terms was even higher (AUC = 0.94; 95% CI: 0.93-0.95).

CONCLUSION

Putative fetal, maternal, and contextual RFs alone fail to achieve reasonable DA for ECS. It is the combination of these RFs and the interactions between them at each hospital that make it possible to improve the DA for the type of delivery and tailor interventions through prediction to improve the appropriateness of ECS indications.

The tyranny of the averages and the indiscriminate use of risk factors in public health: The case of coronary heart disease

Modern medicine is overwhelmed by a plethora of both established risk factors and novel biomarkers for diseases. The majority of this information is expressed by probabilistic measures of association such as the odds ratio (OR) obtained by calculating differences in average “risk” between exposed and unexposed groups. However, recent research demonstrates that even ORs of considerable magnitude are insufficient for assessing the ability of risk factors or biomarkers to distinguish the individuals who will develop the disease from those who will not. In regards to coronary heart disease (CHD), we already know that novel biomarkers add very little to the discriminatory accuracy (DA) of traditional risk factors. However, the value added by traditional risk factors alongside simple demographic variables such as age and sex has been the subject of less discussion. Moreover, in public health, we use the OR to calculate the population attributable fraction (PAF), although this measure fails to consider the DA of the risk factor it represents. Therefore, focusing on CHD and applying measures of DA, we re-examine the role of individual demographic characteristics, risk factors, novel biomarkers and PAFs in public health and epidemiology. In so doing, we also raise a more general criticism of the traditional risk factors’ epidemiology. We investigated a cohort of 6103 men and women who participated in the baseline (1991–1996) of the Malmö Diet and Cancer study and were followed for 18 years. We found that neither traditional risk factors nor biomarkers substantially improved the DA obtained by models considering only age and sex. We concluded that the PAF measure provided insufficient information for the planning of preventive strategies in the population. We need a better understanding of the individual heterogeneity around the averages and, thereby, a fundamental change in the way we interpret risk factors in public health and epidemiology.

•

There is a plethora of differences in “average” risk between exposed and unexposed groups of individuals.

•

Individual heterogeneity around average values is seldom considered in Public Health.

•

Measures of discriminatory accuracy (DA) informs on the underlying individual heterogeneity.

•

Most know risk factors and other categorizations associated with diseases have low DA.

•

We need a fundamental change in the way we investigate risk factors and other categorizations in Public Health.

Revisiting the discriminatory accuracy of traditional risk factors in preeclampsia screening

BACKGROUND

Preeclampsia (PE) is associated with a high risk of perinatal morbidity and mortality. However, there is no consensus in the definition of high-risk women.

AIM

To question current definition of high PE risk and propose a definition that considers individual heterogeneity to improves risk classification.

METHODS

A stratified analysis by parity was conducted using the Swedish Birth Register between 2002-2010 including 626.600 pregnancies. The discriminatory accuracy (DA) of traditional definitions of high-risk women was compared with a new definition based on 1) specific combinations of individual variables and 2) a centile cut-off of the probability of PE predicted by a multiple logistic regression model.

RESULTS

None of the classical risk-factors alone reached an acceptable DA. In multiparous, any combination of a risk-factor with previous PE or HBP reached a +LR>10. The combination of obesity and multiple pregnancy reached a good DA particularly in the presence of previous preeclampsia (positive likelihood ratio (LR+) = 26.5 or chronic hypertension (HBP) LR+ = 40.5. In primiparous, a LR+>15 was observed in multiple pregnancies with the simultaneous presence of obesity and diabetes mellitus or with HBP. Predicted probabilities above 97 centile in multiparous and 99 centile in primiparous provided high (LR+ = 12.5), and moderate (LR+ = 5.85), respectively. No one risk factor alone or in combination provided a LR- sufficiently low to rule-out the disease.

CONCLUSIONS

In preeclampsia prediction the combination of specific risk factors provided a better discriminatory accuracy than traditional single risk approach. Our results contribute to a more personalized risk estimation of preeclampsia.

Absolute rather than relative income is a better socioeconomic predictor of chronic obstructive pulmonary disease in Swedish adults

BACKGROUND

While psychosocial theory claims that socioeconomic status (SES), acting through social comparisons, has an important influence on susceptibility to disease, materialistic theory says that socioeconomic position (SEP) and related access to material resources matter more. However, the relative role of SEP versus SES in chronic obstructive pulmonary disease (COPD) risk has still not been examined.

METHOD

We investigated the association between SES/SEP and COPD risk among 667 094 older adults, aged 55 to 60, residing in Sweden between 2006 and 2011. Absolute income in five groups by population quintiles depicted SEP and relative income expressed as quintile groups within each absolute income group represented SES. We performed sex-stratified logistic regression models to estimate odds ratios and the area under the receiver operator curve (AUC) to compare the discriminatory accuracy of SES and SEP in relation to COPD.

RESULTS

Even though both absolute (SEP) and relative income (SES) were associated with COPD risk, only absolute income (SEP) presented a clear gradient, so the poorest had a three-fold higher COPD risk than the richest individuals. While the AUC for a model including only age was 0.54 and 0.55 when including relative income (SES), it increased to 0.65 when accounting for absolute income (SEP). SEP rather than SES demonstrated a consistent association with COPD.

CONCLUSIONS

Our study supports the materialistic theory. Access to material resources seems more relevant to COPD risk than the consequences of low relative income.

Intersectionality and risk for ischemic heart disease in Sweden: Categorical and anti-categorical approaches

Intersectionality theory can contribute to epidemiology and public health by furthering understanding of power dynamics driving production of health disparities, and increasing knowledge about heterogeneities within, and overlap between, social categories. Drawing on McCall, we relate the first of these potential contributions to categorical intersectionality and the second to anti-categorical intersectionality. Both approaches are used in study of risk of ischemic heart disease (IHD), based on register data on 3.6 million adults residing in Sweden by 2010, followed for three years. Categorical intersectionality is here coupled with between-group differences in average risk calculation, as we use intersectional categorizations while estimating odds ratios through logistic regressions. The anti-categorical approach is operationalized through measurement of discriminatory accuracy (DA), i.e., capacity to accurately categorize individuals with or without a certain outcome, through computation of the area under the curve (AUC). Our results show substantial differences in average risk between intersectional groupings. The DA of social categorizations is found to be low, however, due to outcome variability within and overlap between categories. We argue that measures of DA should be used for proper interpretation of differences in average risk between social (or any other) categories. Tension between average between-group risk and the DA of categorizations, which can be related to categorical and anti-categorical intersectional analyses, should be made explicit and discussed to a larger degree in epidemiology and public health.