Identifying undercompensated groups defined by multiple attributes in risk adjustment

A framework for ex-ante evaluation of the potential effects of risk equalization and risk sharing in health insurance markets with regulated competition

Many health insurance markets are organized by principles of regulated competition. Regulators of these markets typically apply risk equalization (aka risk adjustment) and risk sharing to mitigate risk selection. Risk equalization and risk sharing can have various positive and negative effects on efficiency and fairness. This paper provides a comprehensive framework for ex-ante evaluation of these effects. In a first step, we distinguish 22 potential effects. In a second step, we summarize and discuss quantitative measures used for evaluating risk equalization and risk sharing schemes in academic research. To underline the relevance of our work, we compare our framework with an existing framework that was previously used in the Dutch regulated health insurance market. We conclude that this framework is incomplete and uses inappropriate measures. To avoid suboptimal policy choices, we recommend policymakers (1) to consider the entire spectrum of potential effects and (2) to select their measures carefully.

Scope and Incentives for Risk Selection in Health Insurance Markets With Regulated Competition: A Conceptual Framework and International Comparison

In health insurance markets with regulated competition, regulators face the challenge of preventing risk selection. This paper provides a framework for analyzing the scope (i.e., potential actions by insurers and consumers) and incentives for risk selection in such markets. Our approach consists of three steps. First, we describe four types of risk selection: (a) selection by consumers in and out of the market, (b) selection by consumers between high- and low-value plans, (c) selection by insurers via plan design, and (d) selection by insurers via other channels such as marketing, customer service, and supplementary insurance. In a second step, we develop a conceptual framework of how regulation and features of health insurance markets affect the scope and incentives for risk selection along these four dimensions. In a third step, we use this framework to compare nine health insurance markets with regulated competition in Australia, Europe, Israel, and the United States.

Health equity assessment of machine learning performance (HEAL): a framework and dermatology AI model case study

Background

Artificial intelligence (AI) has repeatedly been shown to encode historical inequities in healthcare. We aimed to develop a framework to quantitatively assess the performance equity of health AI technologies and to illustrate its utility via a case study.

Methods

Here, we propose a methodology to assess whether health AI technologies prioritise performance for patient populations experiencing worse outcomes, that is complementary to existing fairness metrics. We developed the Health Equity Assessment of machine Learning performance (HEAL) framework designed to quantitatively assess the performance equity of health AI technologies via a four-step interdisciplinary process to understand and quantify domain-specific criteria, and the resulting HEAL metric. As an illustrative case study (analysis conducted between October 2022 and January 2023), we applied the HEAL framework to a dermatology AI model. A set of 5420 teledermatology cases (store-and-forward cases from patients of 20 years or older, submitted from primary care providers in the USA and skin cancer clinics in Australia), enriched for diversity in age, sex and race/ethnicity, was used to retrospectively evaluate the AI model's HEAL metric, defined as the likelihood that the AI model performs better for subpopulations with worse average health outcomes as compared to others. The likelihood that AI performance was anticorrelated to pre-existing health outcomes was estimated using bootstrap methods as the probability that the negated Spearman's rank correlation coefficient (i.e., "R") was greater than zero. Positive values of R suggest that subpopulations with poorer health outcomes have better AI model performance. Thus, the HEAL metric, defined as p (R >0), measures how likely the AI technology is to prioritise performance for subpopulations with worse average health outcomes as compared to others (presented as a percentage below). Health outcomes were quantified as disability-adjusted life years (DALYs) when grouping by sex and age, and years of life lost (YLLs) when grouping by race/ethnicity. AI performance was measured as top-3 agreement with the reference diagnosis from a panel of 3 dermatologists per case.

Findings

Across all dermatologic conditions, the HEAL metric was 80.5% for prioritizing AI performance of racial/ethnic subpopulations based on YLLs, and 92.1% and 0.0% respectively for prioritizing AI performance of sex and age subpopulations based on DALYs. Certain dermatologic conditions were significantly associated with greater AI model performance compared to a reference category of less common conditions. For skin cancer conditions, the HEAL metric was 73.8% for prioritizing AI performance of age subpopulations based on DALYs.

Interpretation

Analysis using the proposed HEAL framework showed that the dermatology AI model prioritised performance for race/ethnicity, sex (all conditions) and age (cancer conditions) subpopulations with respect to pre-existing health disparities. More work is needed to investigate ways of promoting equitable AI performance across age for non-cancer conditions and to better understand how AI models can contribute towards improving equity in health outcomes.

Funding

Google LLC.