The effect of conversational agent skill on user behavior during deception

Patient Consent and The Right to Notice and Explanation of AI Systems Used in Health Care

Given the need for enforceable guardrails for artificial intelligence (AI) that protect the public and allow for innovation, the U.S. Government recently issued a Blueprint for an AI Bill of Rights which outlines five principles of safe AI design, use, and implementation. One in particular, the right to notice and explanation, requires accurately informing the public about the use of AI that impacts them in ways that are easy to understand. Yet, in the healthcare setting, it is unclear what goal the right to notice and explanation serves, and the moral importance of patient-level disclosure. We propose three normative functions of this right: (1) to notify patients about their care, (2) to educate patients and promote trust, and (3) to meet standards for informed consent. Additional clarity is needed to guide practices that respect the right to notice and explanation of AI in healthcare while providing meaningful benefits to patients.

Sorting Insiders From Co-Workers: Remote Synchronous Computer-Mediated Triage for Investigating Insider Attacks

OBJECTIVE

Develop and investigate the potential of a remote, computer-mediated and synchronous text-based triage, which we refer to as InSort, for quickly highlighting persons of interest after an insider attack.

BACKGROUND

Insiders maliciously exploit legitimate access to impair the confidentiality and integrity of organizations. The globalisation of organisations and advancement of information technology means employees are often dispersed across national and international sites, working around the clock, often remotely. Hence, investigating insider attacks is challenging. However, the cognitive demands associated with masking insider activity offer opportunities. Drawing on cognitive approaches to deception and understanding of deception-conveying features in textual responses, we developed InSort, a remote computer-mediated triage.

METHOD

During a 6-hour immersive simulation, participants worked in teams, examining password protected, security sensitive databases and exchanging information during an organized crime investigation. Twenty-five percent were covertly incentivized to act as an 'insider' by providing information to a provocateur.

RESULTS

Responses to InSort questioning revealed insiders took longer to answer investigation relevant questions, provided impoverished responses, and their answers were less consistent with known evidence about their behaviours than co-workers.

CONCLUSION

Findings demonstrate InSort has potential to expedite information gathering and investigative processes following an insider attack.

APPLICATION

InSort is appropriate for application by non-specialist investigators and can be quickly altered as a function of both environment and event. InSort offers a clearly defined, well specified, approach for use across insider incidents, and highlights the potential of technology for supporting complex time critical investigations.

Opposing Effects of Response Time in Human–Chatbot Interaction

Research has shown that employing social cues (e.g., name, human-like avatar) in chatbot design enhances users’ social presence perceptions and their chatbot usage intentions. However, the picture is less clear for the social cue of chatbot response time. While some researchers argue that instant responses make chatbots appear unhuman-like, others suggest that delayed responses are perceived less positively. Drawing on social response theory and expectancy violations theory, this study investigates whether users’ prior experience with chatbots clarifies the inconsistencies in the literature. In a lab experiment (N = 202), participants interacted with a chatbot that responded either instantly or with a delay. The results reveal that a delayed response time has opposing effects on social presence and usage intentions and shed light on the differences between novice users and experienced users – that is, those who have not interacted with a chatbot before vs. those who have. This study contributes to information systems literature by identifying prior experience as a key moderating factor that shapes users’ social responses to chatbots and by reconciling inconsistencies in the literature regarding the role of chatbot response time. For practitioners, this study points out a drawback of the widely adopted “one-design-fits-all” approach to chatbot design.

Human-Computer Interaction in Customer Service: The Experience with AI Chatbots—A Systematic Literature Review

Artificial intelligence (AI) conversational agents (CA) or chatbots represent one of the technologies that can provide automated customer service for companies, a trend encountered in recent years. Chatbot use is beneficial for companies when associated with positive customer experience. The purpose of this paper is to analyze the overall customer experience with customer service chatbots in order to identify the main influencing factors for customer experience with customer service chatbots and to identify the resulting dimensions of customer experience (such as perceptions/attitudes and feelings and also responses and behaviors). The analysis uses the systematic literature review (SLR) method and includes a sample of 40 publications that present empirical studies. The results illustrate that the main influencing factors of customer experience with chatbots are grouped in three categories: chatbot-related, customer-related, and context-related factors, where the chatbot-related factors are further categorized in: functional features of chatbots, system features of chatbots and anthropomorphic features of chatbots. The multitude of factors of customer experience result in either positive or negative perceptions/attitudes and feelings of customers. At the same time, customers respond by manifesting their intentions and/or their behaviors towards either the technology itself (chatbot usage continuation and acceptance of chatbot recommendations) or towards the company (buying and recommending products). According to empirical studies, the most influential factors when using chatbots for customer service are response relevance and problem resolution, which usually result in positive customer satisfaction, increased probability for chatbots usage continuation, product purchases, and product recommendations.

Understanding the Design Elements Affecting User Acceptance of Intelligent Agents: Past, Present and Future

Intelligent agents (IAs) are permeating both business and society. However, interacting with IAs poses challenges moving beyond technological limitations towards the human-computer interface. Thus, the knowledgebase related to interaction with IAs has grown exponentially but remains segregated and impedes the advancement of the field. Therefore, we conduct a systematic literature review to integrate empirical knowledge on user interaction with IAs. This is the first paper to examine 107 Information Systems and Human-Computer Interaction papers and identified 389 relationships between design elements and user acceptance of IAs. Along the independent and dependent variables of these relationships, we span a research space model encompassing empirical research on designing for IA user acceptance. Further we contribute to theory, by presenting a research agenda along the dimensions of the research space, which shall be useful to both researchers and practitioners. This complements the past and present knowledge on designing for IA user acceptance with potential pathways into the future of IAs.

Developing Conversational Agents for Use in Criminal Investigations

The adoption of artificial intelligence (AI) systems in environments that involve high risk and high consequence decision-making is severely hampered by critical design issues. These issues include system transparency and brittleness, where transparency relates to (i) the explainability of results and (ii) the ability of a user to inspect and verify system goals and constraints; and brittleness, (iii) the ability of a system to adapt to new user demands. Transparency is a particular concern for criminal intelligence analysis, where there are significant ethical and trust issues that arise when algorithmic and system processes are not adequately understood by a user. This prevents adoption of potentially useful technologies in policing environments. In this article, we present a novel approach to designing a conversational agent (CA) AI system for intelligence analysis that tackles these issues. We discuss the results and implications of three different studies; a Cognitive Task Analysis to understand analyst thinking when retrieving information in an investigation, Emergent Themes Analysis to understand the explanation needs of different system components, and an interactive experiment with a prototype conversational agent. Our prototype conversational agent, named Pan, demonstrates transparency provision and mitigates brittleness by evolving new CA intentions. We encode interactions with the CA with human factors principles for situation recognition and use interactive visual analytics to support analyst reasoning. Our approach enables complex AI systems, such as Pan, to be used in sensitive environments, and our research has broader application than the use case discussed.