Comprehension and engagement in survey interviews with virtual agents

The Effect of Rapport on Data Quality in Face-to-Face Interviews: Beneficial or Detrimental?

The benefits of rapport between interviewers and respondents, in terms of recruiting the latter and motiving them to participate in research, have been generally endorsed. However, there has been less clarity with regard to the association between rapport and data quality. In theory, rapport could be beneficial if it motivates people to give complete and honest responses. On the other hand, efforts to maintain rapport by exhibiting pleasing and socially desirable behaviour could well be detrimental to data quality. In a large longitudinal epidemiological sample, generalized estimating equations (GEE) analyses were used to examine the association between rapport and the following three quality indicators: missing responses, responses to sensitive questions, and consistency of responses. The results of these analyses indicate an association between a high level of rapport and fewer missing responses. In contrast, we found more socially desirable responses for the high-rapport group. Finally, the high-rapport group did not differ from the low-rapport group in terms of the consistency of their responses.

Key changes to improve social presence of a virtual health assistant promoting colorectal cancer screening informed by a technology acceptance model

BACKGROUND

Understanding how older, minoritized patients attend to cues when interacting with web-based health messages may provide opportunities to improve engagement with novel health technologies. We assess acceptance-promoting and acceptance-inhibiting cues of a web-based, intervention promoting colorectal cancer (CRC) screening with a home stool test among Black women.

MATERIALS AND METHODS

Focus group and individual interview data informed iterative changes to a race- and gender-concordant virtual health assistant (VHA). A user-centered design approach was used across 3 iterations to identify changes needed to activate cues described as important; such as portraying authority and expertise. Questionnaire data were analyzed using non-parametric tests for perceptions of cues. Analysis was guided by the Technology Acceptance Model.

RESULTS

Perceptions of interactivity, social presence, expertise, and trust were important cues in a VHA-delivered intervention promoting CRC screening. Features of the web-based platform related to ease of navigation and use were also discussed. Participant comments varied across the 3 iterations and indicated acceptance of or a desire to improve source cues for subsequent iterations. We highlight the specific key changes made at each of three iterative versions of the interactive intervention in conjunction with user perception of changes.

DISCUSSION

Virtual agents can be adapted to better meet patient expectations such as being a trustworthy and expert source. Across three evolving versions of a Black, VHA, cues for social presence were particularly important. Social presence cues helped patients engage with CRC screening messages delivered in this novel digital context.

CONCLUSIONS

When using a VHA to disseminate health information, cues associated with acceptability can be leveraged and adapted as needed for diverse audiences. Patient characteristics (age, identity, health status) are important to note as they may affect perceptions of a novel health technologies ease of use and relevancy according to the leading models.

Can intelligent agents improve data quality in online questiosnnaires? A pilot study

We explored the utility of chatbots for improving data quality arising from collection via sonline surveys. Three-hundred Australian adults sampled via Prolific Academic were randomized across chatbot-supported or unassisted online questionnaire conditions. The questionnaire comprised validated measures, along with challenge items formulated to be confusing yet aligned with the validated targets. The chatbot condition provided optional assistance with item clarity via a virtual support agent. Chatbot use and user satisfaction were measured through session logs and user feedback. Data quality was operationalized as between-group differences in relationships among validated and challenge measures. Findings broadly supported chatbot utility for online surveys, showing that most participants with chatbot access utilized it, found it helpful, and demonstrated modestly improved data quality (vs. controls). Absence of confusion for one challenge item is believed to have contributed to an underestimated effect. Findings show that assistive chatbots can enhance data quality, will be utilized by many participants if available, and are perceived as beneficial by most users. Scope constraints for this pilot study are believed to have led to underestimated effects. Future testing with longer-form questionnaires incorporating expanded item difficulty may further understanding of chatbot utility for survey completion and data quality.

Engagement in Human-Agent Interaction: An Overview

Engagement is a concept of the utmost importance in human-computer interaction, not only for informing the design and implementation of interfaces, but also for enabling more sophisticated interfaces capable of adapting to users. While the notion of engagement is actively being studied in a diverse set of domains, the term has been used to refer to a number of related, but different concepts. In fact it has been referred to across different disciplines under different names and with different connotations in mind. Therefore, it can be quite difficult to understand what the meaning of engagement is and how one study relates to another one accordingly. Engagement has been studied not only in human-human, but also in human-agent interactions i.e., interactions with physical robots and embodied virtual agents. In this overview article we focus on different factors involved in engagement studies, distinguishing especially between those studies that address task and social engagement, involve children and adults, are conducted in a lab or aimed for long term interaction. We also present models for detecting engagement and for generating multimodal behaviors to show engagement.

Self-Guided Web-Based Interventions: Scoping Review on User Needs and the Potential of Embodied Conversational Agents to Address Them

Background

Web-based mental health interventions have evolved from innovative prototypes to evidence-based and clinically applied solutions for mental diseases such as depression and anxiety. Open-access, self-guided types of these solutions hold the promise of reaching and treating a large population at a reasonable cost. However, a considerable factor that currently hinders the effectiveness of these self-guided Web-based interventions is the high level of nonadherence. The absence of a human caregiver apparently has a negative effect on user adherence. It is unknown to what extent this human support can be handed over to the technology of the intervention to mitigate this negative effect.

Objective

The first objective of this paper was to explore what is known in literature about what support a user needs to stay motivated and engaged in an electronic health (eHealth) intervention that requires repeated use. The second objective was to explore the current potential of embodied conversational agents (ECAs) to provide this support.

Methods

This study reviews and interprets the available literature on (1) support within eHealth interventions that require repeated use and (2) the potential of ECAs by means of a scoping review. The rationale for choosing a scoping review is that the subject is broad, diverse, and largely unexplored. Themes for (1) and (2) were proposed based on grounded theory and mapped on each other to find relationships.

Results

The results of the first part of this study suggest the presence of user needs that largely remain implicit and unaddressed. These support needs can be categorized as task-related support and emotion-related support. The results of the second part of this study suggest that ECAs are capable of engaging and motivating users of information technology applications in the domains of learning and behavioral change. Longitudinal studies must be conducted to determine under what circumstances ECAs can create and maintain a productive user relationship. Mapping the user needs on the ECAs’ capabilities suggests that different kinds of ECAs may provide different solutions for improving the adherence levels.

Conclusions

Autonomous ECAs that do not respond to a user’s expressed emotion in real time but take on empathic roles may be sufficient to motivate users to some extent. It is unclear whether those types of ECAs are competent enough and create sufficient believability among users to address the user’s deeper needs for support and empathy. Responsive ECAs may offer a better solution. However, at present, most of these ECAs have difficulties to assess a user’s emotional state in real time during an open dialogue. By conducting future research with relationship theory–based ECAs, the added value of ECAs toward user needs can be better understood.

Mimicry and expressiveness of an ECA in human-agent interaction: familiarity breeds content!

Background

Two experiments investigated the effect of features of human behaviour on the quality of interaction with an Embodied Conversational Agent (ECA).

Methods

In Experiment 1, visual prominence cues (head nod, eyebrow raise) of the ECA were manipulated to explore the hypothesis that likeability of an ECA increases as a function of interpersonal mimicry. In the context of an error detection task, the ECA either mimicked or did not mimic a head nod or brow raise that humans produced to give emphasis to a word when correcting the ECA’s vocabulary. In Experiment 2, presence versus absence of facial expressions on comprehension accuracy of two computer-driven ECA monologues was investigated.

Results

In Experiment 1, evidence for a positive relationship between ECA mimicry and lifelikeness was obtained. However, a mimicking agent did not elicit more human gestures. In Experiment 2, expressiveness was associated with greater comprehension and higher ratings of humour and engagement.

Conclusion

Influences from mimicry can be explained by visual and motor simulation, and bidirectional links between similarity and liking. Cue redundancy and minimizing cognitive load are potential explanations for expressiveness aiding comprehension.

Electronic supplementary material

The online version of this article (doi:10.1186/s40469-016-0008-2) contains supplementary material, which is available to authorized users.