1
|
Lahusen C, Maggetti M, Slavkovik M. Trust, trustworthiness and AI governance. Sci Rep 2024; 14:20752. [PMID: 39237635 DOI: 10.1038/s41598-024-71761-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/18/2024] [Accepted: 08/30/2024] [Indexed: 09/07/2024] Open
Abstract
An emerging issue in AI alignment is the use of artificial intelligence (AI) by public authorities, and specifically the integration of algorithmic decision-making (ADM) into core state functions. In this context, the alignment of AI with the values related to the notions of trust and trustworthiness constitutes a particularly sensitive problem from a theoretical, empirical, and normative perspective. In this paper, we offer an interdisciplinary overview of the scholarship on trust in sociology, political science, and computer science anchored in artificial intelligence. On this basis, we argue that only a coherent and comprehensive interdisciplinary approach making sense of the different properties attributed to trust and trustworthiness can convey a proper understanding of complex watchful trust dynamics in a socio-technical context. Ensuring the trustworthiness of AI-Governance ultimately requires an understanding of how to combine trust-related values while addressing machines, humans and institutions at the same time. We offer a road-map of the steps that could be taken to address the challenges identified.
Collapse
Affiliation(s)
- Christian Lahusen
- Department of Social Sciences, Universität Siegen, 57068, Siegen, Germany
| | - Martino Maggetti
- Université de Lausanne, Institute of Political Studies, CH-1015, Lausanne, Switzerland
| | - Marija Slavkovik
- Information Science and Media Studies, Universitetet I Bergen, 5007, Bergen, Norway.
| |
Collapse
|
2
|
Gall J, Stanton CJ. Low-rank human-like agents are trusted more and blamed less in human-autonomy teaming. Front Artif Intell 2024; 7:1273350. [PMID: 38742120 PMCID: PMC11089226 DOI: 10.3389/frai.2024.1273350] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/06/2023] [Accepted: 04/01/2024] [Indexed: 05/16/2024] Open
Abstract
If humans are to team with artificial teammates, factors that influence trust and shared accountability must be considered when designing agents. This study investigates the influence of anthropomorphism, rank, decision cost, and task difficulty on trust in human-autonomous teams (HAT) and how blame is apportioned if shared tasks fail. Participants (N = 31) completed repeated trials with an artificial teammate using a low-fidelity variation of an air-traffic control game. We manipulated anthropomorphism (human-like or machine-like), military rank of artificial teammates using three-star (superiors), two-star (peers), or one-star (subordinate) agents, the perceived payload of vehicles with people or supplies onboard, and task difficulty with easy or hard missions using a within-subject design. A behavioural measure of trust was inferred when participants accepted agent recommendations, and a measure of no trust when recommendations were rejected or ignored. We analysed the data for trust using binomial logistic regression. After each trial, blame was apportioned using a 2-item scale and analysed using a one-way repeated measures ANOVA. A post-experiment questionnaire obtained participants' power distance orientation using a seven-item scale. Possible power-related effects on trust and blame apportioning are discussed. Our findings suggest that artificial agents with higher levels of anthropomorphism and lower levels of rank increased trust and shared accountability, with human team members accepting more blame for team failures.
Collapse
Affiliation(s)
- Jody Gall
- The MARCS Institute for Brain Behaviour and Development, Western Sydney University, Parramatta, NSW, Australia
| | - Christopher J. Stanton
- The MARCS Institute for Brain Behaviour and Development, Western Sydney University, Parramatta, NSW, Australia
| |
Collapse
|
3
|
Naser MYM, Bhattacharya S. Empowering human-AI teams via Intentional Behavioral Synchrony. FRONTIERS IN NEUROERGONOMICS 2023; 4:1181827. [PMID: 38234496 PMCID: PMC10790871 DOI: 10.3389/fnrgo.2023.1181827] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 03/07/2023] [Accepted: 06/06/2023] [Indexed: 01/19/2024]
Abstract
As Artificial Intelligence (AI) proliferates across various sectors such as healthcare, transportation, energy, and military applications, the collaboration between human-AI teams is becoming increasingly critical. Understanding the interrelationships between system elements - humans and AI - is vital to achieving the best outcomes within individual team members' capabilities. This is also crucial in designing better AI algorithms and finding favored scenarios for joint AI-human missions that capitalize on the unique capabilities of both elements. In this conceptual study, we introduce Intentional Behavioral Synchrony (IBS) as a synchronization mechanism between humans and AI to set up a trusting relationship without compromising mission goals. IBS aims to create a sense of similarity between AI decisions and human expectations, drawing on psychological concepts that can be integrated into AI algorithms. We also discuss the potential of using multimodal fusion to set up a feedback loop between the two partners. Our aim with this work is to start a research trend centered on exploring innovative ways of deploying synchrony between teams of non-human members. Our goal is to foster a better sense of collaboration and trust between humans and AI, resulting in more effective joint missions.
Collapse
Affiliation(s)
- Mohammad Y. M. Naser
- The Neuro-Interaction Innovation Lab, Kennesaw State University, Department of Electrical Engineering, Marietta, GA, United States
| | - Sylvia Bhattacharya
- The Neuro-Interaction Innovation Lab, Kennesaw State University, Department of Engineering Technology, Marietta, GA, United States
| |
Collapse
|
4
|
Song Y, Tao D, Luximon Y. In robot we trust? The effect of emotional expressions and contextual cues on anthropomorphic trustworthiness. APPLIED ERGONOMICS 2023; 109:103967. [PMID: 36736181 DOI: 10.1016/j.apergo.2023.103967] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/05/2022] [Revised: 09/05/2022] [Accepted: 01/09/2023] [Indexed: 06/18/2023]
Abstract
Following the evolution of technology and its application in various daily contexts, social robots work as an advanced artificial intelligence (AI) system to interact with humans. However, limited research has been done to discuss the role of emotional expressions and contextual cues in influencing anthropomorphic trustworthiness, especially from the design perspective. To address this research gap, the current study designed a specific robot prototype and conducted two lab experiments to explore the effect of emotional expressions and contextual cues on trustworthiness via a combination of subjective ratings and physiological measures. Results showed that: 1) positive (vs. negative) emotional expressions enjoyed a higher level of anthropomorphic trustworthiness and visual attention; 2) regulatory fit was expanded in parasocial interaction and worked as a prime to activate anthropomorphic trustworthiness for social robots. Theoretical contributions and design implications were also discussed in this study.
Collapse
Affiliation(s)
- Yao Song
- College of Literature and Journalism, Sichuan University, Chengdu, China; Convergence Laboratory of Chinese Cultural Inheritance and Global Communication, Sichuan University, Chengdu, China; School of Design, The Hong Kong Polytechnic University, Hung Hom, Hong Kong Special Administrative Region of China
| | - Da Tao
- Institute of Human Factors and Ergonomics, College of Mechatronics and Control Engineering, Shenzhen University, Shenzhen, China
| | - Yan Luximon
- School of Design, The Hong Kong Polytechnic University, Hung Hom, Hong Kong Special Administrative Region of China.
| |
Collapse
|
5
|
Bobko P, Hirshfield L, Eloy L, Spencer C, Doherty E, Driscoll J, Obolsky H. Human-agent teaming and trust calibration: a theoretical framework, configurable testbed, empirical illustration, and implications for the development of adaptive systems. THEORETICAL ISSUES IN ERGONOMICS SCIENCE 2022. [DOI: 10.1080/1463922x.2022.2086644] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/17/2022]
Affiliation(s)
| | | | - Lucca Eloy
- University of Colorado at Boulder, Boulder, CO, USA
| | - Cara Spencer
- University of Colorado at Boulder, Boulder, CO, USA
| | | | | | | |
Collapse
|
6
|
Maier T, Menold J, McComb C. The Relationship Between Performance and Trust in AI in E-Finance. Front Artif Intell 2022; 5:891529. [PMID: 35800065 PMCID: PMC9253559 DOI: 10.3389/frai.2022.891529] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/07/2022] [Accepted: 05/30/2022] [Indexed: 11/27/2022] Open
Abstract
Artificial intelligence (AI) is fundamentally changing how people work in nearly every field, including online finance. However, our ability to interact with AI is moderated by factors such as performance, complexity, and trust. The work presented in this study analyzes the effect of performance on trust in a robo-advisor (AI which assists in managing investments) through an empirical investment simulation. Results show that for applications where humans and AI have comparable capabilities, the difference in performance (between the human and AI) is a moderate indicator of change in trust; however, human or AI performance individually were weak indicators. Additionally, results indicate that biases typically seen in human-human interactions may also occur in human-AI interactions when AI transparency is low.
Collapse
Affiliation(s)
- Torsten Maier
- Department of Industrial and Manufacturing Engineering, Kettering University, Flint, MI, United States
| | - Jessica Menold
- School of Engineering Design, Technology, and Professional Programs, The Pennsylvania State University, University Park, State College, PA, United States
| | - Christopher McComb
- Department of Mechanical Engineering, Carnegie Mellon University, Pittsburgh, PA, United States
- *Correspondence: Christopher McComb
| |
Collapse
|
7
|
Trust and ethics in AI. AI & SOCIETY 2022. [DOI: 10.1007/s00146-022-01473-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/18/2022]
|
8
|
Hopko SK, Mehta RK. Neural Correlates of Trust in Automation: Considerations and Generalizability Between Technology Domains. FRONTIERS IN NEUROERGONOMICS 2021; 2:731327. [PMID: 38235218 PMCID: PMC10790920 DOI: 10.3389/fnrgo.2021.731327] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 06/26/2021] [Accepted: 08/10/2021] [Indexed: 01/19/2024]
Abstract
Investigations into physiological or neurological correlates of trust has increased in popularity due to the need for a continuous measure of trust, including for trust-sensitive or adaptive systems, measurements of trustworthiness or pain points of technology, or for human-in-the-loop cyber intrusion detection. Understanding the limitations and generalizability of the physiological responses between technology domains is important as the usefulness and relevance of results is impacted by fundamental characteristics of the technology domains, corresponding use cases, and socially acceptable behaviors of the technologies. While investigations into the neural correlates of trust in automation has grown in popularity, there is limited understanding of the neural correlates of trust, where the vast majority of current investigations are in cyber or decision aid technologies. Thus, the relevance of these correlates as a deployable measure for other domains and the robustness of the measures to varying use cases is unknown. As such, this manuscript discusses the current-state-of-knowledge in trust perceptions, factors that influence trust, and corresponding neural correlates of trust as generalizable between domains.
Collapse
Affiliation(s)
- Sarah K. Hopko
- Neuroergonomics Lab, Department of Industrial and Systems Engineering, Texas A&M University, College Station, TX, United States
| | | |
Collapse
|
9
|
Song Y, Luximon A, Luximon Y. The effect of facial features on facial anthropomorphic trustworthiness in social robots. APPLIED ERGONOMICS 2021; 94:103420. [PMID: 33823378 DOI: 10.1016/j.apergo.2021.103420] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/24/2020] [Revised: 12/05/2020] [Accepted: 03/21/2021] [Indexed: 06/12/2023]
Abstract
As the nature of human-robot relationships have become increasingly bound to shift from supervisor-machine to friend-companion, people have exhibited an increasing interest in making social judgments toward such anthropomorphic objects, such as trustworthiness. However, the facial features of social robots and their potential effect on anthropomorphic trustworthiness are seldom analyzed and discussed comprehensively. This study examined whether the trustworthiness perception toward a social robot shared similarity with baby schema features on the human face. It also explored the effects of different combinations of baby schema facial features, especially the positions and sizes of the eyes and mouth, on facial anthropomorphic trustworthiness. A 5-way mixed experiment (N = 270) was conducted accordingly. The results indicated that people would experience a high level of facial anthropomorphic trustworthiness toward robots with baby schema features (i.e., large eyes, with medium vertical and horizontal positions of the eyes and mouth). This paper contributes to the literature on facial anthropomorphic trustworthiness in human-robot interaction and provides suggestions for social robot design.
Collapse
Affiliation(s)
- Yao Song
- School of Design, The Hong Kong Polytechnic University, Hung Hom, Hong Kong, China.
| | | | - Yan Luximon
- School of Design, The Hong Kong Polytechnic University, Hung Hom, Hong Kong, China.
| |
Collapse
|
10
|
Alarcon GM, Gibson AM, Jessup SA, Capiola A. Exploring the differential effects of trust violations in human-human and human-robot interactions. APPLIED ERGONOMICS 2021; 93:103350. [PMID: 33529968 DOI: 10.1016/j.apergo.2020.103350] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/01/2020] [Revised: 12/21/2020] [Accepted: 12/22/2020] [Indexed: 06/12/2023]
Abstract
There is sparse research directly investigating the effects of trust manipulations in human-human and human-robot interactions. Moreover, studies on human-human versus human-robot trust have leveraged unusual or low vulnerability contexts to investigate such effects and have focused mostly on robot performance. In the present research, we seek to remedy these limitations and compare trust in human-human versus human-robot collaborations in an augmented and adapted version of the Trust Game. We used a mixed factorial design to examine the effects of trust and trust violations on human-human and human-robot interactions over time with an emphasis on anthropomorphic robots in a social context. We found consistent and significant effects of partner behavior. Specifically, partner distrust behaviors led to participants' lower levels of trustworthiness perceptions, trust intentions, and trust behaviors over time compared to partner trust behaviors. We found no significant effect of partnering with a human versus an anthropomorphic robot over time across the three dependent variables, supporting the computers as social actors (CASA; Nass and Moon, 2000) paradigm. This study demonstrated that there may be instances where the effects of trust violations from an anthropomorphized robot partner are not meaningfully different from those of a human partner in a social context.
Collapse
Affiliation(s)
- Gene M Alarcon
- Air Force Research Laboratory, 2210 Eighth Street Bldg. 146, Wright Patterson Air Force Base, OH, 45433, USA.
| | - Anthony M Gibson
- Consortium of Universities, 1020 19th St NW #500, Washington, DC, 20036, USA.
| | - Sarah A Jessup
- Air Force Research Laboratory, 2210 Eighth Street Bldg. 146, Wright Patterson Air Force Base, OH, 45433, USA.
| | - August Capiola
- Air Force Research Laboratory, 2210 Eighth Street Bldg. 146, Wright Patterson Air Force Base, OH, 45433, USA.
| |
Collapse
|
11
|
|
12
|
Kim W, Kim N, Lyons JB, Nam CS. Factors affecting trust in high-vulnerability human-robot interaction contexts: A structural equation modelling approach. APPLIED ERGONOMICS 2020; 85:103056. [PMID: 32174344 DOI: 10.1016/j.apergo.2020.103056] [Citation(s) in RCA: 12] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/22/2018] [Revised: 11/28/2019] [Accepted: 01/13/2020] [Indexed: 06/10/2023]
Abstract
The current research proposed and tested a structural equation model (SEM) that describes hypothesized relationships among factors affecting trust in human-robot interaction (HRI) such as trustworthiness, human-likeness, intelligence, perfect automation schema (PAS), and affect. A video stimulus depicting an autonomous guard robot interacting with humans was employed as a stimulus via Amazon's Mechanical Turk to recruit 233 participants. Human-related and robot-related metrics were found to affect trustworthiness that subsequently affected trust. In particular, ability (as a trustworthiness facet) was a dominant factor affecting trust in HRI. Integrity was found to mediate the relationships between robot- and human-related metrics and trustworthiness. This study also showed a correlation between intelligence and trustworthiness, as well as between PAS and trustworthiness. The findings of the present study have significant implications for both theory and practice on factors and levels that affect trust in HRI.
Collapse
Affiliation(s)
- Wonjoon Kim
- Department of Industrial & Management Engineering, Sungkyul University, South Korea
| | - Nayoung Kim
- Edward P. Fitts Department of Industrial & Systems Engineering, North Carolina State University, USA
| | | | - Chang S Nam
- Edward P. Fitts Department of Industrial & Systems Engineering, North Carolina State University, USA.
| |
Collapse
|
13
|
Affiliation(s)
- Guido Möllering
- Reinhard Mohn Institute of Management, Witten/Herdecke University, Witten, Germany
| |
Collapse
|