The Development of Overtrust: An Empirical Simulation and Psychological Analysis in the Context of Human-Robot Interaction

Existing Barriers Faced by and Future Design Recommendations for Direct-to-Consumer Health Care Artificial Intelligence Apps: Scoping Review

BACKGROUND

Direct-to-consumer (DTC) health care artificial intelligence (AI) apps hold the potential to bridge the spatial and temporal disparities in health care resources, but they also come with individual and societal risks due to AI errors. Furthermore, the manner in which consumers interact directly with health care AI is reshaping traditional physician-patient relationships. However, the academic community lacks a systematic comprehension of the research overview for such apps.

OBJECTIVE

This paper systematically delineated and analyzed the characteristics of included studies, identified existing barriers and design recommendations for DTC health care AI apps mentioned in the literature and also provided a reference for future design and development.

METHODS

This scoping review followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews guidelines and was conducted according to Arksey and O'Malley's 5-stage framework. Peer-reviewed papers on DTC health care AI apps published until March 27, 2023, in Web of Science, Scopus, the ACM Digital Library, IEEE Xplore, PubMed, and Google Scholar were included. The papers were analyzed using Braun and Clarke's reflective thematic analysis approach.

RESULTS

Of the 2898 papers retrieved, 32 (1.1%) covering this emerging field were included. The included papers were recently published (2018-2023), and most (23/32, 72%) were from developed countries. The medical field was mostly general practice (8/32, 25%). In terms of users and functionalities, some apps were designed solely for single-consumer groups (24/32, 75%), offering disease diagnosis (14/32, 44%), health self-management (8/32, 25%), and health care information inquiry (4/32, 13%). Other apps connected to physicians (5/32, 16%), family members (1/32, 3%), nursing staff (1/32, 3%), and health care departments (2/32, 6%), generally to alert these groups to abnormal conditions of consumer users. In addition, 8 barriers and 6 design recommendations related to DTC health care AI apps were identified. Some more subtle obstacles that are particularly worth noting and corresponding design recommendations in consumer-facing health care AI systems, including enhancing human-centered explainability, establishing calibrated trust and addressing overtrust, demonstrating empathy in AI, improving the specialization of consumer-grade products, and expanding the diversity of the test population, were further discussed.

CONCLUSIONS

The booming DTC health care AI apps present both risks and opportunities, which highlights the need to explore their current status. This paper systematically summarized and sorted the characteristics of the included studies, identified existing barriers faced by, and made future design recommendations for such apps. To the best of our knowledge, this is the first study to systematically summarize and categorize academic research on these apps. Future studies conducting the design and development of such systems could refer to the results of this study, which is crucial to improve the health care services provided by DTC health care AI apps.

Invisible but Understandable: In Search of the Sweet Spot between Technology Invisibility and Transparency in Smart Spaces and Beyond

Smart technology is already present in many areas of everyday life. People rely on algorithms in crucial life domains such as finance and healthcare, and the smart car promises a more relaxed driving experience—all the while, the technology recedes further into the background. The smarter the technology, the more intransparent it tends to become. Users no longer understand how the technology works, what its limits are, and what consequences regarding autonomy and privacy emerge. Both extremes, total invisibility and total transparency, come with specific challenges and do not form reasonable design goals. This research explores the potential tension between smart and invisible versus transparent and understandable technology. We discuss related theories from the fields of explainable AI (XAI) as well as trust psychology, and then introduce transparency in smart spaces as a special field of application. A case study explores specific challenges and design approaches through the example of a so-called room intelligence (RI), i.e., a special kind of smart living room. We conclude with research perspectives for more general design approaches and implications for future research.

Multi-device trust transfer: Can trust be transferred among multiple devices?

Recent advances in automation technology have increased the opportunity for collaboration between humans and multiple autonomous systems such as robots and self-driving cars. In research on autonomous system collaboration, the trust users have in autonomous systems is an important topic. Previous research suggests that the trust built by observing a task can be transferred to other tasks. However, such research did not focus on trust in multiple different devices but in one device or several of the same devices. Thus, we do not know how trust changes in an environment involving the operation of multiple different devices such as a construction site. We investigated whether trust can be transferred among multiple different devices, and investigated the effect of two factors: the similarity among multiple devices and the agency attributed to each device, on trust transfer among multiple devices. We found that the trust a user has in a device can be transferred to other devices and that attributing different agencies to each device can clarify the distinction among devices, preventing trust from transferring.

Finite-Time Interactive Control of Robots with Multiple Interaction Modes

This paper proposes a finite-time multi-modal robotic control strategy for physical human-robot interaction. The proposed multi-modal controller consists of a modified super-twisting-based finite-time control term that is designed in each interaction mode and a continuity-guaranteed control term. The finite-time control term guarantees finite-time achievement of the desired impedance dynamics in active interaction mode (AIM), makes the tracking error of the reference trajectory converge to zero in finite time in passive interaction mode (PIM), and also guarantees robotic motion stop in finite time in safety-stop mode (SSM). Meanwhile, the continuity-guaranteed control term guarantees control input continuity and steady interaction modes transition. The finite-time closed-loop control stability and the control effectiveness is validated by Lyapunov-based theoretical analysis and simulations on a robot manipulator.

Human Control Model Estimation in Physical Human-Machine Interaction: A Survey

The study of human-machine interaction as a unique control system was one of the first research interests in the engineering field, with almost a century having passed since the first works appeared in this area. At the same time, it is a crucial aspect of the most recent technological developments made in application fields such as collaborative robotics and artificial intelligence. Learning the processes and dynamics underlying human control strategies when interacting with controlled elements or objects of a different nature has been the subject of research in neuroscience, aerospace, robotics, and artificial intelligence. The cross-domain nature of this field of study can cause difficulties in finding a guiding line that links motor control theory, modelling approaches in physiological control systems, and identifying human-machine general control models in manipulative tasks. The discussed models have varying levels of complexity, from the first quasi-linear model in the frequency domain to the successive optimal control model. These models include detailed descriptions of physiologic subsystems and biomechanics. The motivation behind this work is to provide a complete view of the linear models that could be easily handled both in the time domain and in the frequency domain by using a well-established methodology in the classical linear systems and control theory.

Expect the Unexpected: Leveraging the Human-Robot Ecosystem to Handle Unexpected Robot Failures

Unexpected robot failures are inevitable. We propose to leverage socio-technical relations within the human-robot ecosystem to support adaptable strategies for handling unexpected failures. The Theory of Graceful Extensibility is used to understand how characteristics of the ecosystem can influence its ability to respond to unexpected events. By expanding our perspective from Human-Robot Interaction to the Human-Robot Ecosystem, adaptable failure-handling strategies are identified, alongside technical, social and organizational arrangements that are needed to support them. We argue that robotics and HRI communities should pursue more holistic approaches to failure-handling, recognizing the need to embrace the unexpected and consider socio-technical relations within the human robot ecosystem when designing failure-handling strategies.

Can Robots Earn Our Trust the Same Way Humans Do? A Systematic Exploration of Competence, Warmth, and Anthropomorphism as Determinants of Trust Development in HRI

Robots increasingly act as our social counterparts in domains such as healthcare and retail. For these human-robot interactions (HRI) to be effective, a question arises on whether we trust robots the same way we trust humans. We investigated whether the determinants competence and warmth, known to influence interpersonal trust development, influence trust development in HRI, and what role anthropomorphism plays in this interrelation. In two online studies with 2 × 2 between-subjects design, we investigated the role of robot competence (Study 1) and robot warmth (Study 2) in trust development in HRI. Each study explored the role of robot anthropomorphism in the respective interrelation. Videos showing an HRI were used for manipulations of robot competence (through varying gameplay competence) and robot anthropomorphism (through verbal and non-verbal design cues and the robot's presentation within the study introduction) in Study 1 (n = 155) as well as robot warmth (through varying compatibility of intentions with the human player) and robot anthropomorphism (same as Study 1) in Study 2 (n = 157). Results show a positive effect of robot competence (Study 1) and robot warmth (Study 2) on trust development in robots regarding anticipated trust and attributed trustworthiness. Subjective perceptions of competence (Study 1) and warmth (Study 2) mediated the interrelations in question. Considering applied manipulations, robot anthropomorphism neither moderated interrelations of robot competence and trust (Study 1) nor robot warmth and trust (Study 2). Considering subjective perceptions, perceived anthropomorphism moderated the effect of perceived competence (Study 1) and perceived warmth (Study 2) on trust on an attributional level. Overall results support the importance of robot competence and warmth for trust development in HRI and imply transferability regarding determinants of trust development in interpersonal interaction to HRI. Results indicate a possible role of perceived anthropomorphism in these interrelations and support a combined consideration of these variables in future studies. Insights deepen the understanding of key variables and their interaction in trust dynamics in HRI and suggest possibly relevant design factors to enable appropriate trust levels and a resulting desirable HRI. Methodological and conceptual limitations underline benefits of a rather robot-specific approach for future research.