Trustworthy artificial intelligence and the European Union AI act: On the conflation of trustworthiness and acceptability of risk

Artificial intelligence in neurovascular decision-making: a comparative analysis of ChatGPT-4 and multidisciplinary expert recommendations for unruptured intracranial aneurysms

In the multidisciplinary treatment of cerebrovascular diseases, specialists from different disciplines strive to develop patient-specific treatment recommendations. ChatGPT is a natural language processing chatbot with increasing applicability in medical practice. This study evaluates ChatGPT's ability to provide treatment recommendations for patients with unruptured intracranial aneurysms (UIA). Anonymized patient data and radiological reports of 20 patients with UIAs were provided to GPT-4 in a standardized format and used to generate a treatment recommendation for different clinical scenarios. GPT-4 responses were evaluated by a multidisciplinary panel of specialists by means of the Likert scale and subsequently benchmarked against the Unruptured Intracranial Aneurysm Treatment Score (UIATS) as well as the actual treatment decision made by the multidisciplinary institutional neurovascular board (INVB). Agreement between expert raters was measured using linear weighted Fleiss-Kappa coefficient. GPT-4 analyzed individual pathological features of the radiological reports and formulated a corresponding assessment for each aspect. None of the recommendations generated reflected evidence of factual hallucination, although in 25% of the case studies no specific recommendation could be derived from the GPT-4 responses. The expert panel rated the overall quality of the GPT-4 recommendations with a median of 3.4 out of 5 points. The GPT-4 recommendations were congruent with those of the INBI in 65% of cases. Interrater reliability among experts showed moderate to low agreement in the assessment of AI-assisted decision making. GPT-4 appears to be able to process clinical information about UIAs and generate treatment recommendations. However, the level of ambiguity and the utilization of scientific evidence in the recommendations are not yet patient/case specific enough to substitute the decision-making of a multidisciplinary neurovascular board. A prospective evaluation of GPT-4 competence as a companion in decision-making panels is deemed necessary.

Bias in adjudication: Investigating the impact of artificial intelligence, media, financial and legal institutions in pursuit of social justice

The latest global progress report highlights numerous challenges in achieving justice goals, with bias in artificial intelligence (AI) emerging as a significant yet underexplored issue. This paper investigates the role of AI in addressing bias within the judicial system to promote equitable social justice. Analyzing weekly data from January 1, 2019, to December 31, 2023, through wavelet quantile correlation, this study examines the short, medium, and long-term impacts of integrating AI, media, international legal influence (ILI), and international financial institutions (IFI) as crucial factors in achieving Sustainable Development Goal 16 (SDG-16), which focuses on justice. The findings indicate that AI, media, ILI, and IFI can help reduce bias in the medium and long term, although their effects appear mixed and less significant in the short term. Our research proposes a comprehensive policy framework that addresses the complexities of implementing these technologies in the judicial system. We conclude that successfully integrating AI requires a supportive global policy environment that embraces technological innovation, financial backing, and robust regulation to prevent potential disruptions that could reinforce inequalities, perpetuate structural injustices, and exacerbate human rights issues, ultimately leading to more biased outcomes in social justice.

A roadmap for improving data quality through standards for collaborative intelligence in human-robot applications

Collaborative intelligence (CI) involves human-machine interactions and is deemed safety-critical because their reliable interactions are crucial in preventing severe injuries and environmental damage. As these applications become increasingly data-driven, the reliability of CI applications depends on the quality of data, shaping the system's ability to interpret and respond in diverse and often unpredictable environments. In this regard, it is important to adhere to data quality standards and guidelines, thus facilitating the advancement of these collaborative systems in industry. This study presents the challenges of data quality in CI applications within industrial environments, with two use cases that focus on the collection of data in Human-Robot Interaction (HRI). The first use case involves a framework for quantifying human and robot performance within the context of naturalistic robot learning, wherein humans teach robots using intuitive programming methods within the domain of HRI. The second use case presents real-time user state monitoring for adaptive multi-modal teleoperation, that allows for a dynamic adaptation of the system's interface, interaction modality and automation level based on user needs. The article proposes a hybrid standardization derived from established data quality-related ISO standards and addresses the unique challenges associated with multi-modal HRI data acquisition. The use cases presented in this study were carried out as part of an EU-funded project, Collaborative Intelligence for Safety-Critical Systems (CISC).

Justifying Our Credences in the Trustworthiness of AI Systems: A Reliabilistic Approach

We address an open problem in the philosophy of artificial intelligence (AI): how to justify the epistemic attitudes we have towards the trustworthiness of AI systems. The problem is important, as providing reasons to believe that AI systems are worthy of trust is key to appropriately rely on these systems in human-AI interactions. In our approach, we consider the trustworthiness of an AI as a time-relative, composite property of the system with two distinct facets. One is the actual trustworthiness of the AI and the other is the perceived trustworthiness of the system as assessed by its users while interacting with it. We show that credences, namely, beliefs we hold with a degree of confidence, are the appropriate attitude for capturing the facets of the trustworthiness of an AI over time. Then, we introduce a reliabilistic account providing justification to the credences in the trustworthiness of AI, which we derive from Tang's probabilistic theory of justified credence. Our account stipulates that a credence in the trustworthiness of an AI system is justified if and only if it is caused by an assessment process that tends to result in a high proportion of credences for which the actual and perceived trustworthiness of the AI are calibrated. This approach informs research on the ethics of AI and human-AI interactions by providing actionable recommendations on how to measure the reliability of the process through which users perceive the trustworthiness of the system, investigating its calibration to the actual levels of trustworthiness of the AI as well as users' appropriate reliance on the system.

Digital (mis)trust: ethnographic encounters with computational forms

This special issue is driven by a curiosity about the role of computational forms - practices, logics, technologies, and infrastructures - in social life and how they mediate issues of trust and mistrust: designated (mis)trust. Through a series of ethnographic encounters, its contributors describe how (mis)trust is rendered as an issue of concern by various actors as it is problematized, conceptualized, narrated, and designed for. In doing so, the papers analyse the work (mis)trust does in these settings, with a focus on the role of computational thinking within public discourses on democratic elections, the use of computational technologies in establishing bureaucratic order, the computational practices at play in the production of coding subjectivities, and the computational artefacts that assure data circulations within digital infrastructures. This introduction argues for a more expansive understanding of the relation between trust and mistrust in the digital age, countering the oftentimes default rendering of these concepts as antonymic. Instead, it argues that they live in a mutable relation. Despite prevailing technosolutionist approaches to (mis)trust, it cannot, we suggest, be solved for, resolved, or even eviscerated. Whatever actors (engineers, programmers, professionals etc) do in their efforts to 'fix' mistrust, it continues to mutate as a social form.

AI in Radiology: Navigating Medical Responsibility

The application of Artificial Intelligence (AI) facilitates medical activities by automating routine tasks for healthcare professionals. AI augments but does not replace human decision-making, thus complicating the process of addressing legal responsibility. This study investigates the legal challenges associated with the medical use of AI in radiology, analyzing relevant case law and literature, with a specific focus on professional liability attribution. In the case of an error, the primary responsibility remains with the physician, with possible shared liability with developers according to the framework of medical device liability. If there is disagreement with the AI's findings, the physician must not only pursue but also justify their choices according to prevailing professional standards. Regulations must balance the autonomy of AI systems with the need for responsible clinical practice. Effective use of AI-generated evaluations requires knowledge of data dynamics and metrics like sensitivity and specificity, even without a clear understanding of the underlying algorithms: the opacity (referred to as the "black box phenomenon") of certain systems raises concerns about the interpretation and actual usability of results for both physicians and patients. AI is redefining healthcare, underscoring the imperative for robust liability frameworks, meticulous updates of systems, and transparent patient communication regarding AI involvement.

How the EU AI Act Seeks to Establish an Epistemic Environment of Trust

With focus on the development and use of artificial intelligence (AI) systems in the digital health context, we consider the following questions: How does the European Union (EU) seek to facilitate the development and uptake of trustworthy AI systems through the AI Act? What does trustworthiness and trust mean in the AI Act, and how are they linked to some of the ongoing discussions of these terms in bioethics, law, and philosophy? What are the normative components of trustworthiness? And how do the requirements of the AI Act relate to these components? We first explain how the EU seeks to create an epistemic environment of trust through the AI Act to facilitate the development and uptake of trustworthy AI systems. The legislation establishes a governance regime that operates as a socio-epistemological infrastructure of trust which enables a performative framing of trust and trustworthiness. The degree of success that performative acts of trust and trustworthiness have achieved in realising the legislative goals may then be assessed in terms of statutorily defined proxies of trustworthiness. We show that to be trustworthy, these performative acts should be consistent with the ethical principles endorsed by the legislation; these principles are also manifested in at least four key features of the governance regime. However, specified proxies of trustworthiness are not expected to be adequate for applications of AI systems within a regulatory sandbox or in real-world testing. We explain why different proxies of trustworthiness for these applications may be regarded as 'special' trust domains and why the nature of trust should be understood as participatory.

Medical Ethics and Artificial Intelligence in Neurosurgery-How Should We Prepare?

BACKGROUND

The development of artificial intelligence (AI) raises ethical concerns about its side effects on the attitudes and behaviors of clinicians and medical practitioners. The authors aim to understand the medical ethics of AI-based chatbots and to suggest coping strategies for an emerging landscape of increased access and potential ambiguity using AI.

METHODS

This study examines the medical ethics of AI-based chatbots (Chat generative pretrained transformer [GPT], Bing Chat, and Google's Bard) using multiple-choice questions. ChatGPT and Bard correctly answered all questions (5/5), while Bing Chat correctly answered only 3 of 5 questions. ChatGPT explained answers simply. Bing Chat explained answers with references, and Bard provided additional explanations with details.

RESULTS

AI has the potential to revolutionize medical fields by improving diagnosis accuracy, surgical planning, and treatment outcomes. By analyzing large amounts of data, AI can identify patterns and make predictions, aiding neurosurgeons in making informed decisions for increased patient wellbeing. As AI usage increases, the number of cases involving AI-entrusted judgments will rise, leading to the gradual emergence of ethical issues across interdisciplinary fields. The medical field will be no exception.

CONCLUSIONS

This study suggests the need for safety measures to regulate medical ethics in the context of advancing AI. A system should be developed to verify and predict pertinent issues.

How Will Artificial Intelligence Shape the Future of Decision-Making in Congenital Heart Disease?

Improvements in medical technology have significantly changed the management of congenital heart disease (CHD), offering novel tools to predict outcomes and personalize follow-up care. By using sophisticated imaging modalities, computational models and machine learning algorithms, clinicians can experiment with unprecedented insights into the complex anatomy and physiology of CHD. These tools enable early identification of high-risk patients, thus allowing timely, tailored interventions and improved outcomes. Additionally, the integration of genetic testing offers valuable prognostic information, helping in risk stratification and treatment optimisation. The birth of telemedicine platforms and remote monitoring devices facilitates customised follow-up care, enhancing patient engagement and reducing healthcare disparities. Taking into consideration challenges and ethical issues, clinicians can make the most of the full potential of artificial intelligence (AI) to further refine prognostic models, personalize care and improve long-term outcomes for patients with CHD. This narrative review aims to provide a comprehensive illustration of how AI has been implemented as a new technological method for enhancing the management of CHD.

Orbital learning: a novel, actively orchestrated decentralised learning for healthcare

A novel collaborative and continual learning across a network of decentralised healthcare units, avoiding identifiable data-sharing capacity, is proposed. Currently available methodologies, such as federated learning and swarm learning, have demonstrated decentralised learning. However, the majority of them face shortcomings that affect their performance and accuracy. These shortcomings include a non-uniform rate of data accumulation, non-uniform patient demographics, biased human labelling, and erroneous or malicious training data. A novel method to reduce such shortcomings is proposed in the present work through selective grouping and displacing of actors in a network of many entities for intra-group sharing of learning with inter-group accessibility. The proposed system, known as Orbital Learning, incorporates various features from split learning and ensemble learning for a robust and secure performance of supervised models. A digital embodiment of the information quality and flow within a decentralised network, this platform also acts as a digital twin of healthcare network. An example of ECG classification for arrhythmia with 6 clients is used to analyse its performance and is compared against federated learning. In this example, four separate experiments are conducted with varied configurations, such as varied age demographics and clients with data tampering. The results obtained show an average area under receiver operating characteristic curve (AUROC) of 0.819 (95% CI 0.784-0.853) for orbital learning whereas 0.714 (95% CI 0.692-0.736) for federated learning. This result shows an increase in overall performance and establishes that the proposed system can address the majority of the issues faced by existing decentralised learning methodologies. Further, a scalability demo conducted establishes the versatility and scalability of this platform in handling state-of-the-art large language models.

Managing the race to the moon: Global policy and governance in Artificial Intelligence regulation—A contemporary overview and an analysis of socioeconomic consequences

This paper delves into the complexities of global AI regulation and governance, emphasizing the socio-economic repercussions of rapid AI development. It scrutinizes the challenges in creating effective governance structures amidst the AI race, considering diverse global perspectives and policies. The discourse moves beyond specific corporate examples, addressing broader implications and sector-wide impacts of AI on employment, truth discernment, and democratic stability. The analysis focuses on contrasting regulatory approaches across key regions—the United States, European Union, Asia, Africa, and the Americas and thus highlighting the variations and commonalities in strategies and implementations. This comparative study reveals the intricacies and hurdles in formulating a cohesive global policy for AI regulation. Central to the paper is the examination of the dynamic between rapid AI innovation and the slower pace of regulatory and ethical standard-setting. It critically evaluates the advantages and drawbacks of shifting regulatory responsibilities between government bodies and the private sector. In response to these challenges, the discussion proposes an innovative and integrated regulatory model. The model advocates for a collaborative network that blends governmental authority with industry expertise, aiming to establish adaptive, responsive regulations (called “dynamic laws”) that can evolve with technological advancements. The novel approach aims to bridge the gap between rapid AI advancements in the industry and the essential democratic processes of law-making.

Institutionalised distrust and human oversight of artificial intelligence: towards a democratic design of AI governance under the European Union AI Act

Human oversight has become a key mechanism for the governance of artificial intelligence ("AI"). Human overseers are supposed to increase the accuracy and safety of AI systems, uphold human values, and build trust in the technology. Empirical research suggests, however, that humans are not reliable in fulfilling their oversight tasks. They may be lacking in competence or be harmfully incentivised. This creates a challenge for human oversight to be effective. In addressing this challenge, this article aims to make three contributions. First, it surveys the emerging laws of oversight, most importantly the European Union's Artificial Intelligence Act ("AIA"). It will be shown that while the AIA is concerned with the competence of human overseers, it does not provide much guidance on how to achieve effective oversight and leaves oversight obligations for AI developers underdefined. Second, this article presents a novel taxonomy of human oversight roles, differentiated along whether human intervention is constitutive to, or corrective of a decision made or supported by an AI. The taxonomy allows to propose suggestions for improving effectiveness tailored to the type of oversight in question. Third, drawing on scholarship within democratic theory, this article formulates six normative principles which institutionalise distrust in human oversight of AI. The institutionalisation of distrust has historically been practised in democratic governance. Applied for the first time to AI governance, the principles anticipate the fallibility of human overseers and seek to mitigate them at the level of institutional design. They aim to directly increase the trustworthiness of human oversight and to indirectly inspire well-placed trust in AI governance.