Moral Engagement and Disengagement in Health Care AI Development

BACKGROUND

Machine learning (ML) is utilized increasingly in health care, and can pose harms to patients, clinicians, health systems, and the public. In response, regulators have proposed an approach that would shift more responsibility to ML developers for mitigating potential harms. To be effective, this approach requires ML developers to recognize, accept, and act on responsibility for mitigating harms. However, little is known regarding the perspectives of developers themselves regarding their obligations to mitigate harms.

METHODS

We conducted 40 semi-structured interviews with developers of ML predictive analytics applications for health care in the United States.

RESULTS

Participants varied widely in their perspectives on personal responsibility and included examples of both moral engagement and disengagement, albeit in a variety of forms. While most (70%) of participants made a statement indicative of moral engagement, most of these statements reflected an awareness of moral issues, while only a subset of these included additional elements of engagement such as recognizing responsibility, alignment with personal values, addressing conflicts of interests, and opportunities for action. Further, we identified eight distinct categories of moral disengagement reflecting efforts to minimize potential harms or deflect personal responsibility for preventing or mitigating harms.

CONCLUSIONS

These findings suggest possible facilitators and barriers to the development of ethical ML that could act by encouraging moral engagement or discouraging moral disengagement. Regulatory approaches that depend on the ability of ML developers to recognize, accept, and act on responsibility for mitigating harms might have limited success without education and guidance for ML developers about the extent of their responsibilities and how to implement them.

The clinical utility of three frailty measures in identifying HIV-associated neurocognitive disorders

OBJECTIVE

Frailty measures vary widely and the optimal measure for predicting HIV-associated neurocognitive disorders (HAND) is unclear.

DESIGN

A study was conducted to examine the clinical utility of three widely used frailty measures in identifying HIV-associated neurocognitive disorders.

METHODS

The study involved 284 people with HIV (PWH) at least 50 years enrolled at UC San Diego's HIV Neurobehavioral Research Program. Frailty measurements included the Fried Phenotype, the Rockwood Frailty Index, and the Veterans Aging Cohort Study (VACS) Index. HAND was diagnosed according to Frascati criteria. ANOVAs examined differences in frailty severity across HAND conditions. ROC analyses evaluated sensitivity and specificity of each measure to detect symptomatic HAND [mild neurocognitive disorder (MND) and HIV-associated dementia (HAD)] from no HAND.

RESULTS

Across all frailty measures, frailty was found to be higher in HAD compared with no HAND. For Fried and Rockwood (not VACS), frailty was significantly more severe in MND vs. no HAND and in HAD vs. ANI (asymptomatic neurocognitive impairment). For discriminating symptomatic HAND from no HAND, Fried was 37% sensitive and 92% specific, Rockwood was 85% sensitive and 43% specific, and VACS was 58% sensitive and 65% specific.

CONCLUSION

These findings demonstrate that Fried and Rockwood outperform VACS in predicting HAND. However, ROC analyses suggest none of the indices had adequate predictive validity in detecting HAND. The results indicate that the combined use of the Rockwood and Fried indices may be an appropriate alternative.

Developer Perspectives on Potential Harms of Machine Learning Predictive Analytics in Health Care: Qualitative Analysis

BACKGROUND

Machine learning predictive analytics (MLPA) is increasingly used in health care to reduce costs and improve efficacy; it also has the potential to harm patients and trust in health care. Academic and regulatory leaders have proposed a variety of principles and guidelines to address the challenges of evaluating the safety of machine learning-based software in the health care context, but accepted practices do not yet exist. However, there appears to be a shift toward process-based regulatory paradigms that rely heavily on self-regulation. At the same time, little research has examined the perspectives about the harms of MLPA developers themselves, whose role will be essential in overcoming the "principles-to-practice" gap.

OBJECTIVE

The objective of this study was to understand how MLPA developers of health care products perceived the potential harms of those products and their responses to recognized harms.

METHODS

We interviewed 40 individuals who were developing MLPA tools for health care at 15 US-based organizations, including data scientists, software engineers, and those with mid- and high-level management roles. These 15 organizations were selected to represent a range of organizational types and sizes from the 106 that we previously identified. We asked developers about their perspectives on the potential harms of their work, factors that influence these harms, and their role in mitigation. We used standard qualitative analysis of transcribed interviews to identify themes in the data.

RESULTS

We found that MLPA developers recognized a range of potential harms of MLPA to individuals, social groups, and the health care system, such as issues of privacy, bias, and system disruption. They also identified drivers of these harms related to the characteristics of machine learning and specific to the health care and commercial contexts in which the products are developed. MLPA developers also described strategies to respond to these drivers and potentially mitigate the harms. Opportunities included balancing algorithm performance goals with potential harms, emphasizing iterative integration of health care expertise, and fostering shared company values. However, their recognition of their own responsibility to address potential harms varied widely.

CONCLUSIONS

Even though MLPA developers recognized that their products can harm patients, public, and even health systems, robust procedures to assess the potential for harms and the need for mitigation do not exist. Our findings suggest that, to the extent that new oversight paradigms rely on self-regulation, they will face serious challenges if harms are driven by features that developers consider inescapable in health care and business environments. Furthermore, effective self-regulation will require MLPA developers to accept responsibility for safety and efficacy and know how to act accordingly. Our results suggest that, at the very least, substantial education will be necessary to fill the "principles-to-practice" gap.

Rapid Review of COVID-19 Vaccination Access and Acceptance for Global Refugee, Asylum Seeker and Undocumented Migrant Populations

Objectives: Refugees, asylum seekers, and undocumented migrants globally have been disproportionally impacted by COVID-19. Vaccination has been a major tool to reduce disease impact, yet concerns exist regarding equitable allocation and uptake. Methods: A rapid literature review was conducted based on PRISMA guidelines to determine COVID-19 vaccination acceptance rates and level of access for these population groups globally. Results: Relatively high COVID-19 vaccine acceptance levels were commonly reported in these populations, although, trust in host governments was a frequently expressed concern, especially for undocumented migrants. Outreach efforts and access to comprehensive information from a trusted source and in appropriate language were found to be major determinants of COVID-19 vaccine acceptance. COVID-19 vaccination access and policies varied considerably across host countries despite urgings by international organizations to include migrants and refugees. While most governments endorsed inclusive policies, evidence of successful program implementation was frequently lacking, creating difficulty to better tailor and implement COVID-19 outreach programs. Conclusion: This review identifies impactful improvements to be implemented to ensure equitable COVID-19 vaccinations and to reduce disease burden on refugees, asylum seekers, and undocumented migrants.

Diverse experts' perspectives on ethical issues of using machine learning to predict HIV/AIDS risk in sub-Saharan Africa: a modified Delphi study

OBJECTIVE

To better understand diverse experts' views about the ethical implications of ongoing research funded by the National Institutes of Health that uses machine learning to predict HIV/AIDS risk in sub-Saharan Africa (SSA) based on publicly available Demographic and Health Surveys data.

DESIGN

Three rounds of semi-structured surveys in an online expert panel using a modified Delphi approach.

PARTICIPANTS

Experts in informatics, African public health and HIV/AIDS and bioethics were invited to participate.

MEASURES

Perceived importance of or agreement about relevance of ethical issues on 5-point unipolar Likert scales. Qualitative data analysis identified emergent themes related to ethical issues and development of an ethical framework and recommendations for open-ended questions.

RESULTS

Of the 35 invited experts, 22 participated in the online expert panel (63%). Emergent themes were the inclusion of African researchers in all aspects of study design, analysis and dissemination to identify and address local contextual issues, as well as engagement of communities. Experts focused on engagement with health and science professionals to address risks, benefits and communication of findings. Respondents prioritised the mitigation of stigma to research participants but recognised trade-offs between privacy and the need to disseminate findings to realise public health benefits. Strategies for responsible communication of results were suggested, including careful word choice in presentation of results and limited dissemination to need-to-know stakeholders such as public health planners.

CONCLUSION

Experts identified ethical issues specific to the African context and to research on sensitive, publicly available data and strategies for addressing these issues. These findings can be used to inform an ethical implementation framework with research stage-specific recommendations on how to use publicly available data for machine learning-based predictive analytics to predict HIV/AIDS risk in SSA.

A Typology of Existing Machine Learning-Based Predictive Analytic Tools Focused on Reducing Costs and Improving Quality in Health Care: Systematic Search and Content Analysis

BACKGROUND

Considerable effort has been devoted to the development of artificial intelligence, including machine learning-based predictive analytics (MLPA) for use in health care settings. The growth of MLPA could be fueled by payment reforms that hold health care organizations responsible for providing high-quality, cost-effective care. Policy analysts, ethicists, and computer scientists have identified unique ethical and regulatory challenges from the use of MLPA in health care. However, little is known about the types of MLPA health care products available on the market today or their stated goals.

OBJECTIVE

This study aims to better characterize available MLPA health care products, identifying and characterizing claims about products recently or currently in use in US health care settings that are marketed as tools to improve health care efficiency by improving quality of care while reducing costs.

METHODS

We conducted systematic database searches of relevant business news and academic research to identify MLPA products for health care efficiency meeting our inclusion and exclusion criteria. We used content analysis to generate MLPA product categories and characterize the organizations marketing the products.

RESULTS

We identified 106 products and characterized them based on publicly available information in terms of the types of predictions made and the size, type, and clinical training of the leadership of the companies marketing them. We identified 5 categories of predictions made by MLPA products based on publicly available product marketing materials: disease onset and progression, treatment, cost and utilization, admissions and readmissions, and decompensation and adverse events.

CONCLUSIONS

Our findings provide a foundational reference to inform the analysis of specific ethical and regulatory challenges arising from the use of MLPA to improve health care efficiency.

Ethics in Research: Relevance for Nephrology

Research is crucial to progress in nephrology. It is important that studies are conducted rigorously from the scientific perspective, as well as in adherence to ethical standards. Traditional clinical research places a high value on individual research subject autonomy. Research questions often include the clinical effectiveness of new interventions studied under highly controlled conditions. Such research has brought the promise of new game-changers in nephrology, such as the sodium-glucose cotransporter 2 inhibitors. Implementation research takes such knowledge further and investigates how to translate it into broader-scale policy and practice, to achieve swift and global uptake, with a focus on justice and equity. Newer challenges arising globally in research ethics include those relating to oversight of innovation, biobanking and big data, human-challenge studies, and research during emergencies. This article details the history of clinical research ethics and the role of research ethics committees, describes the evolving spectrum of biomedical research in human medicine, and presents emerging clinical research ethics issues using illustrative examples and a hypothetical case study. It is imperative that researchers and research ethics committees are well versed in the ethical principles of all forms of human research such that research is conducted to the highest standards and that effective interventions can be implemented at scale as rapidly as possible.

Ethics of emerging infectious disease outbreak responses: Using Ebola virus disease as a case study of limited resource allocation

Emerging infectious diseases such as Ebola Virus Disease (EVD), Nipah Virus Encephalitis and Lassa fever pose significant epidemic threats. Responses to emerging infectious disease outbreaks frequently occur in resource-constrained regions and under high pressure to quickly contain the outbreak prior to potential spread. As seen in the 2020 EVD outbreaks in the Democratic Republic of Congo and the current COVID-19 pandemic, there is a continued need to evaluate and address the ethical challenges that arise in the high stakes environment of an emerging infectious disease outbreak response. The research presented here provides analysis of the ethical challenges with regard to allocation of limited resources, particularly experimental therapeutics, using the 2013–2016 EVD outbreak in West Africa as a case study. In-depth semi-structured interviews were conducted with senior healthcare personnel (n = 16) from international humanitarian aid organizations intimately engaged in the 2013–2016 EVD outbreak response in West Africa. Interviews were recorded in private setting, transcribed, and iteratively coded using grounded theory methodology. A majority of respondents indicated a clear propensity to adopt an ethical framework of guiding principles for international responses to emerging infectious disease outbreaks. Respondents agreed that prioritization of frontline workers’ access to experimental therapeutics was warranted based on a principle of reciprocity. There was widespread acceptance of adaptive trial designs and greater trial transparency in providing access to experimental therapeutics. Many respondents also emphasized the importance of community engagement in limited resource allocation scheme design and culturally appropriate informed consent procedures. The study results inform a potential ethical framework of guiding principles based on the interview participants’ insights to be adopted by international response organizations and their healthcare workers in the face of allocating limited resources such as experimental therapeutics in future emerging infectious disease outbreaks to ease the moral burden of individual healthcare providers.