The Potential For Bias In Machine Learning And Opportunities For Health Insurers To Address It

Bridging Health Disparities in the Data-Driven World of Artificial Intelligence: A Narrative Review

BACKGROUND

Artificial intelligence (AI) holds exciting potential to revolutionize healthcare delivery in the United States. However, there are concerns about its potential to perpetuate disparities among historically marginalized populations.

OBJECTIVE

Following the guidelines of Preferred Reporting Items for Systematic Reviews and Meta-Analyses, we conducted a narrative review of current literature on AI and health disparities in the United States. We aimed to answer the question, Does AI have the potential to reduce or eliminate health disparities, or will its use further exacerbate these disparities?

METHODS

We searched the Ovid MEDLINE electronic database to identify and retrieve publications discussing AI and its impact on racial/ethnic health disparities. Articles were included if they discussed AI as a tool to mitigate racial health disparities with or without bias in developing and using AI.

RESULTS

This review included 65 articles. We identified six themes of limitations in AI that impact health equity: (1) biases in AI can perpetuate and exacerbate racial and ethnic inequities; (2) equity in algorithms should be a priority; (3) lack of diversity in the field of AI is concerning; (4) the need for regulation and testing algorithms for accuracy; (5) ethical standards for AI in health care are needed; and (6) the importance of promoting transparency and accountability.

CONCLUSIONS

While AI promises to enhance healthcare outcomes and address equity concerns, risks and challenges are associated with its implementation. To maximize the use of AI, it must be approached with an equity lens during all phases of development.

Artificial intelligence/machine learning for epilepsy and seizure diagnosis

Accurate seizure and epilepsy diagnosis remains a challenging task due to the complexity and variability of manifestations, which can lead to delayed or missed diagnosis. Machine learning (ML) and artificial intelligence (AI) is a rapidly developing field, with growing interest in integrating and applying these tools to aid clinicians facing diagnostic uncertainties. ML algorithms, particularly deep neural networks, are increasingly employed in interpreting electroencephalograms (EEG), neuroimaging, wearable data, and seizure videos. This review discusses the development and testing phases of AI/ML tools, emphasizing the importance of generalizability and interpretability in medical applications, and highlights recent publications that demonstrate the current and potential utility of AI to aid clinicians in diagnosing epilepsy. Current barriers of AI integration in patient care include dataset availability and heterogeneity, which limit studies' quality, interpretability, comparability, and generalizability. ML and AI offer substantial promise in improving the accuracy and efficiency of epilepsy diagnosis. The growing availability of diverse datasets, enhanced processing speed, and ongoing efforts to standardize reporting contribute to the evolving landscape of AI applications in clinical care.

The Sociodemographic Biases in Machine Learning Algorithms: A Biomedical Informatics Perspective

Artificial intelligence models represented in machine learning algorithms are promising tools for risk assessment used to guide clinical and other health care decisions. Machine learning algorithms, however, may house biases that propagate stereotypes, inequities, and discrimination that contribute to socioeconomic health care disparities. The biases include those related to some sociodemographic characteristics such as race, ethnicity, gender, age, insurance, and socioeconomic status from the use of erroneous electronic health record data. Additionally, there is concern that training data and algorithmic biases in large language models pose potential drawbacks. These biases affect the lives and livelihoods of a significant percentage of the population in the United States and globally. The social and economic consequences of the associated backlash cannot be underestimated. Here, we outline some of the sociodemographic, training data, and algorithmic biases that undermine sound health care risk assessment and medical decision-making that should be addressed in the health care system. We present a perspective and overview of these biases by gender, race, ethnicity, age, historically marginalized communities, algorithmic bias, biased evaluations, implicit bias, selection/sampling bias, socioeconomic status biases, biased data distributions, cultural biases and insurance status bias, conformation bias, information bias and anchoring biases and make recommendations to improve large language model training data, including de-biasing techniques such as counterfactual role-reversed sentences during knowledge distillation, fine-tuning, prefix attachment at training time, the use of toxicity classifiers, retrieval augmented generation and algorithmic modification to mitigate the biases moving forward.

Translating Predictive Analytics for Public Health Practice: A Case Study of Overdose Prevention in Rhode Island

Prior applications of machine learning to population health have relied on conventional model assessment criteria, limiting the utility of models as decision support tools for public health practitioners. To facilitate practitioners' use of machine learning as a decision support tool for area-level intervention, we developed and applied 4 practice-based predictive model evaluation criteria (implementation capacity, preventive potential, health equity, and jurisdictional practicalities). We used a case study of overdose prevention in Rhode Island to illustrate how these criteria could inform public health practice and health equity promotion. We used Rhode Island overdose mortality records from January 2016-June 2020 (n = 1,408) and neighborhood-level US Census data. We employed 2 disparate machine learning models, Gaussian process and random forest, to illustrate the comparative utility of our criteria to guide interventions. Our models predicted 7.5%-36.4% of overdose deaths during the test period, illustrating the preventive potential of overdose interventions assuming 5%-20% statewide implementation capacities for neighborhood-level resource deployment. We describe the health equity implications of use of predictive modeling to guide interventions along the lines of urbanicity, racial/ethnic composition, and poverty. We then discuss considerations to complement predictive model evaluation criteria and inform the prevention and mitigation of spatially dynamic public health problems across the breadth of practice. This article is part of a Special Collection on Mental Health.

A clinical classification framework for identifying persons with high social and medical needs: The COMPLEXedex-SDH

BACKGROUND

First-generation algorithms resulted in high-cost features as a representation of need but unintentionally introduced systemic bias based on prior ability to access care. Improved precision health approaches are needed to reduce bias and improve health equity.

PURPOSE

To integrate nursing expertise into a clinical definition of high-need cases and develop a clinical classification algorithm for implementing nursing interventions.

METHODS

Two-phase retrospective, descriptive cohort study using 2019 data to build the algorithm (n = 19,20,848) and 2021 data to test it in adults ≥18 years old (n = 15,99,176).

DISCUSSION

The COMPLEXedex-SDH algorithm identified the following populations: cross-cohort needs (10.9%); high-need persons (cross-cohort needs and other social determinants) (17.7%); suboptimal health care utilization for persons with medical complexity (13.8%); high need persons with suboptimal health care utilization (6.2%).

CONCLUSION

The COMPLEXedex-SDH enables the identification of high-need cases and value-based utilization into actionable cohorts to prioritize outreach calls to improve health equity and outcomes.

Contextual Forces Shaping Home-Based Health Care Services Between 2010 and 2020: Insights From the Social-Ecological Model and Organizational Theory

Demands for home-based care have surged alongside population aging, preferences for aging in place, policy-driven reforms incentivizing lower hospital utilization, and public concerns around coronavirus disease 2019 transmissions in institutional care settings. However, at both macro and micro levels, sociopolitical, and infrastructural contexts are not aligned with the operational needs of home health care organizations, presenting obstacles to home health care equity. We integrate the social-ecological model and organizational theory to highlight contextual forces shaping the delivery of home-based care services between 2010 and 2020. Placing home-based health care organizations at the center of observation, we discuss patterns and trends of service delivery as systematic organizational behaviors reflecting the organizations' adaptations and responses to their surrounding forces. In this light, we consider the implications of provision and access to home care services for health equity, discuss topics that are understudied, and provide recommendations for home-based health care organizations to advance home health care equity. The article represents a synthesis of recent literature and our research and industry experiences.

Assessing performance of ZCTA-level and Census Tract-level social and environmental risk factors in a model predicting hospital events

Predictive analytics are used in primary care to efficiently direct health care resources to high-risk patients to prevent unnecessary health care utilization and improve health. Social determinants of health (SDOH) are important features in these models, but they are poorly measured in administrative claims data. Area-level SDOH can be proxies for unavailable individual-level indicators, but the extent to which the granularity of risk factors impacts predictive models is unclear. We examined whether increasing the granularity of area-based SDOH features from ZIP code tabulation area (ZCTA) to Census Tract strengthened an existing clinical prediction model for avoidable hospitalizations (AH events) in Maryland Medicare fee-for-service beneficiaries. We created a person-month dataset for 465,749 beneficiaries (59.4% female; 69.8% White; 22.7% Black) with 144 features indexing medical history and demographics using Medicare claims (September 2018 through July 2021). Claims data were linked with 37 SDOH features associated with AH events from 11 publicly-available sources (e.g., American Community Survey) based on the beneficiaries' ZCTA and Census Tract of residence. Individual AH risk was estimated using six discrete time survival models with different combinations of demographic, condition/utilization, and SDOH features. Each model used stepwise variable selection to retain only meaningful predictors. We compared model fit, predictive performance, and interpretation across models. Results showed that increasing the granularity of area-based risk factors did not dramatically improve model fit or predictive performance. However, it did affect model interpretation by altering which SDOH features were retained during variable selection. Further, the inclusion of SDOH at either granularity level meaningfully reduced the risk that was attributed to demographic predictors (e.g., race, dual-eligibility for Medicaid). Differences in interpretation are critical given that this model is used by primary care staff to inform the allocation of care management resources, including those available to address drivers of health beyond the bounds of traditional health care.

Designing Equitable Health Care Outreach Programs From Machine Learning Patient Risk Scores

There is growing interest in ensuring equity and guarding against bias in the use of risk scores produced by machine learning and artificial intelligence models. Risk scores are used to select patients who will receive outreach and support. Inappropriate use of risk scores, however, can perpetuate disparities. Commonly advocated solutions to improve equity are nontrivial to implement and may not pass legal scrutiny. In this article, we introduce pragmatic tools that support better use of risk scores for more equitable outreach programs. Our model output charts allow modeling and care management teams to see the equity consequences of different threshold choices and to select the optimal risk thresholds to trigger outreach. For best results, as with any health equity tool, we recommend that these charts be used by a diverse team and shared with relevant stakeholders.

A Survey of Research Participants’ Privacy-Related Experiences and Willingness to Share Real-World Data with Researchers

Background: Real-world data (RWD) privacy is an increasingly complex topic within the scope of personalized medicine, as it implicates several sources of data. Objective: To assess how privacy-related experiences, when adjusted for age and education level, may shape adult research participants’ willingness to share various sources of real-world data with researchers. Methods: An electronic survey was conducted in April 2021 among adults (≥18 years of age) registered in ResearchMatch, a national health research registry. Descriptive analyses were conducted to assess survey participant demographics. Logistic regression was conducted to assess the association between participants’ five distinct privacy-related experiences and their willingness to share each of the 19 data sources with researchers, adjusting for education level and age range. Results: A total of 598 ResearchMatch adults were contacted and 402 completed the survey. Most respondents were over the age of 51 years (49% total) and held a master’s or bachelor’s degree (63% total). Over half of participants (54%) had their account accessed by someone without their permission. Almost half of participants (49%) reported the privacy of their personal information being violated. Analyses showed that, when adjusted for age range and education level, participants whose reputations were negatively affected as a result of information posted online were more likely to share electronic medical record data (OR = 2.074, 95% CI: 0.986–4.364) and genetic data (OR = 2.302, 95% CI: 0.894–5.93) versus those without this experience. Among participants who had an unpleasant experience as a result of giving out information online, those with some college/associates/trade school compared to those with a doctoral or other terminal degree were significantly more willing to share genetic data (OR = 1.064, 95% CI: 0.396–2.857). Across all privacy-related experiences, participants aged 18 to 30 were significantly more likely than those over 60 years to share music streaming data, ridesharing history data, and voting history data. Additionally, across all privacy-related experiences, those with a high school education were significantly more likely than those with a doctorate or other terminal degree to share credit card statement data. Conclusions: This study offers the first insights into how privacy-related experiences, adjusted for age range and education level, may shape ResearchMatch participants’ willingness to share several sources of real-world data sources with precision medicine researchers. Future work should further explore these insights.

A “Do No Harm” Novel Safety Checklist and Research Approach to Determine Whether to Launch an Artificial Intelligence Based Medical Technology – Introducing the Biological-Psychological, Economic, and Social Framework (Preprint)

Given the impact artificial intelligence (AI)-based medical technologies (hardware devices, software programs, and mobile apps) can have on society, debates regarding the principles behind their development and deployment are emerging. Using the biopsychosocial model applied in psychiatry and other fields of medicine as our foundation, we propose a novel 3-step framework to guide industry developers of AI-based medical tools as well as health care regulatory agencies on how to decide if a product should be launched-a "Go or No-Go" approach. More specifically, our novel framework places stakeholders' (patients, health care professionals, industry, and government institutions) safety at its core by asking developers to demonstrate the biological-psychological (impact on physical and mental health), economic, and social value of their AI tool before it is launched. We also introduce a novel cost-effective, time-sensitive, and safety-oriented mixed quantitative and qualitative clinical phased trial approach to help industry and government health care regulatory agencies test and deliberate on whether to launch these AI-based medical technologies. To our knowledge, our biological-psychological, economic, and social (BPES) framework and mixed method phased trial approach are the first to place the Hippocratic Oath of "Do No Harm" at the center of developers', implementers', regulators', and users' mindsets when determining whether an AI-based medical technology is safe to launch. Moreover, as the welfare of AI users and developers becomes a greater concern, our framework's novel safety feature will allow it to complement existing and future AI reporting guidelines.