A Call for an Ethics and Governance Action Plan to Harness the Power of Artificial Intelligence and Digitalization in Nephrology

How the EU AI Act Seeks to Establish an Epistemic Environment of Trust

With focus on the development and use of artificial intelligence (AI) systems in the digital health context, we consider the following questions: How does the European Union (EU) seek to facilitate the development and uptake of trustworthy AI systems through the AI Act? What does trustworthiness and trust mean in the AI Act, and how are they linked to some of the ongoing discussions of these terms in bioethics, law, and philosophy? What are the normative components of trustworthiness? And how do the requirements of the AI Act relate to these components? We first explain how the EU seeks to create an epistemic environment of trust through the AI Act to facilitate the development and uptake of trustworthy AI systems. The legislation establishes a governance regime that operates as a socio-epistemological infrastructure of trust which enables a performative framing of trust and trustworthiness. The degree of success that performative acts of trust and trustworthiness have achieved in realising the legislative goals may then be assessed in terms of statutorily defined proxies of trustworthiness. We show that to be trustworthy, these performative acts should be consistent with the ethical principles endorsed by the legislation; these principles are also manifested in at least four key features of the governance regime. However, specified proxies of trustworthiness are not expected to be adequate for applications of AI systems within a regulatory sandbox or in real-world testing. We explain why different proxies of trustworthiness for these applications may be regarded as 'special' trust domains and why the nature of trust should be understood as participatory.

Investigating the Impact of AI on Shared Decision-Making in Post-Kidney Transplant Care (PRIMA-AI): Protocol for a Randomized Controlled Trial

BACKGROUND

Patients after kidney transplantation eventually face the risk of graft loss with the concomitant need for dialysis or retransplantation. Choosing the right kidney replacement therapy after graft loss is an important preference-sensitive decision for kidney transplant recipients. However, the rate of conversations about treatment options after kidney graft loss has been shown to be as low as 13% in previous studies. It is unknown whether the implementation of artificial intelligence (AI)-based risk prediction models can increase the number of conversations about treatment options after graft loss and how this might influence the associated shared decision-making (SDM).

OBJECTIVE

This study aims to explore the impact of AI-based risk prediction for the risk of graft loss on the frequency of conversations about the treatment options after graft loss, as well as the associated SDM process.

METHODS

This is a 2-year, prospective, randomized, 2-armed, parallel-group, single-center trial in a German kidney transplant center. All patients will receive the same routine post-kidney transplant care that usually includes follow-up visits every 3 months at the kidney transplant center. For patients in the intervention arm, physicians will be assisted by a validated and previously published AI-based risk prediction system that estimates the risk for graft loss in the next year, starting from 3 months after randomization until 24 months after randomization. The study population will consist of 122 kidney transplant recipients >12 months after transplantation, who are at least 18 years of age, are able to communicate in German, and have an estimated glomerular filtration rate <30 mL/min/1.73 m2. Patients with multi-organ transplantation, or who are not able to communicate in German, as well as underage patients, cannot participate. For the primary end point, the proportion of patients who have had a conversation about their treatment options after graft loss is compared at 12 months after randomization. Additionally, 2 different assessment tools for SDM, the CollaboRATE mean score and the Control Preference Scale, are compared between the 2 groups at 12 months and 24 months after randomization. Furthermore, recordings of patient-physician conversations, as well as semistructured interviews with patients, support persons, and physicians, are performed to support the quantitative results.

RESULTS

The enrollment for the study is ongoing. The first results are expected to be submitted for publication in 2025.

CONCLUSIONS

This is the first study to examine the influence of AI-based risk prediction on physician-patient interaction in the context of kidney transplantation. We use a mixed methods approach by combining a randomized design with a simple quantitative end point (frequency of conversations), different quantitative measurements for SDM, and several qualitative research methods (eg, records of physician-patient conversations and semistructured interviews) to examine the implementation of AI-based risk prediction in the clinic.

TRIAL REGISTRATION

ClinicalTrials.gov NCT06056518; https://clinicaltrials.gov/study/NCT06056518.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

PRR1-10.2196/54857.

Ethical Implications of Chatbot Utilization in Nephrology

This comprehensive critical review critically examines the ethical implications associated with integrating chatbots into nephrology, aiming to identify concerns, propose policies, and offer potential solutions. Acknowledging the transformative potential of chatbots in healthcare, responsible implementation guided by ethical considerations is of the utmost importance. The review underscores the significance of establishing robust guidelines for data collection, storage, and sharing to safeguard privacy and ensure data security. Future research should prioritize defining appropriate levels of data access, exploring anonymization techniques, and implementing encryption methods. Transparent data usage practices and obtaining informed consent are fundamental ethical considerations. Effective security measures, including encryption technologies and secure data transmission protocols, are indispensable for maintaining the confidentiality and integrity of patient data. To address potential biases and discrimination, the review suggests regular algorithm reviews, diversity strategies, and ongoing monitoring. Enhancing the clarity of chatbot capabilities, developing user-friendly interfaces, and establishing explicit consent procedures are essential for informed consent. Striking a balance between automation and human intervention is vital to preserve the doctor-patient relationship. Cultural sensitivity and multilingual support should be considered through chatbot training. To ensure ethical chatbot utilization in nephrology, it is imperative to prioritize the development of comprehensive ethical frameworks encompassing data handling, security, bias mitigation, informed consent, and collaboration. Continuous research and innovation in this field are crucial for maximizing the potential of chatbot technology and ultimately improving patient outcomes.

Operationalizing "One Health" as "One Digital Health" Through a Global Framework That Emphasizes Fair and Equitable Sharing of Benefits From the Use of Artificial Intelligence and Related Digital Technologies

The operationalization of One Health (OH) through digitalization is a means to deploy digital technologies (including Artificial Intelligence (AI), big data and related digital technologies) to better capacitate us to deal with growing climate exigency and related threats to human, animal and plant health. With reference to the concept of One Digital Health (ODH), this paper considers how digital capabilities can help to overcome ‘operational brakes’ in OH through new and deeper insights, better predictions, and more targeted or precise preventive strategies and public health countermeasures. However, the data landscape is fragmented and access to certain types of data is increasingly restrictive as individuals, communities and countries seek to assert greater control over data taken from them. This paper proposes for a dedicated global ODH framework—centered on fairness and equity—to be established to promote data-sharing across all the key knowledge domains of OH and to devise data-driven solutions to challenges in the human-animal-ecosystems interface. It first considers the data landscape in relation to: (1) Human and population health; (2) Pathogens; (3) Animal and plant health; and (4) Ecosystems and biodiversity. The complexification from the application of advance genetic sequencing technology is then considered, with focus on current debates over whether certain types of data like digital (genetic) sequencing information (DSI) should remain openly and freely accessible. The proposed ODH framework must augment the existing access and benefit sharing (ABS) framework currently prescribed under the Nagoya Protocol to the Convention on Biological Diversity (CBD) in at least three different ways. First, the ODH framework should apply to all genetic resources and data, including DSI, whether from humans or non-humans. Second, the FAIRER principles should be implemented, with focus on fair and equitable benefit-sharing. Third, the ODH framework should adopt multilateral approaches to data sharing (such as through federated data systems) and to ABS. By operationalizing OH as ODH, we are more likely to be able to protect and restore natural habitats, secure the health and well-being of all living things, and thereby realize the goals set out in the post-2020 Global Biodiversity Framework under the CBD.