Virtual conversational agents versus online forms: Patient experience and preferences for health data collection

Usability, Engagement, and Report Usefulness of Chatbot-Based Family Health History Data Collection: Mixed Methods Analysis

BACKGROUND

Family health history (FHx) is an important predictor of a person's genetic risk but is not collected by many adults in the United States.

OBJECTIVE

This study aims to test and compare the usability, engagement, and report usefulness of 2 web-based methods to collect FHx.

METHODS

This mixed methods study compared FHx data collection using a flow-based chatbot (KIT; the curious interactive test) and a form-based method. KIT's design was optimized to reduce user burden. We recruited and randomized individuals from 2 crowdsourced platforms to 1 of the 2 FHx methods. All participants were asked to complete a questionnaire to assess the method's usability, the usefulness of a report summarizing their experience, user-desired chatbot enhancements, and general user experience. Engagement was studied using log data collected by the methods. We used qualitative findings from analyzing free-text comments to supplement the primary quantitative results.

RESULTS

Participants randomized to KIT reported higher usability than those randomized to the form, with a mean System Usability Scale score of 80.2 versus 61.9 (P<.001), respectively. The engagement analysis reflected design differences in the onboarding process. KIT users spent less time entering FHx information and reported more conditions than form users (mean 5.90 vs 7.97 min; P=.04; and mean 7.8 vs 10.1 conditions; P=.04). Both KIT and form users somewhat agreed that the report was useful (Likert scale ratings of 4.08 and 4.29, respectively). Among desired enhancements, personalization was the highest-rated feature (188/205, 91.7% rated medium- to high-priority). Qualitative analyses revealed positive and negative characteristics of both KIT and the form-based method. Among respondents randomized to KIT, most indicated it was easy to use and navigate and that they could respond to and understand user prompts. Negative comments addressed KIT's personality, conversational pace, and ability to manage errors. For KIT and form respondents, qualitative results revealed common themes, including a desire for more information about conditions and a mutual appreciation for the multiple-choice button response format. Respondents also said they wanted to report health information beyond KIT's prompts (eg, personal health history) and for KIT to provide more personalized responses.

CONCLUSIONS

We showed that KIT provided a usable way to collect FHx. We also identified design considerations to improve chatbot-based FHx data collection: First, the final report summarizing the FHx collection experience should be enhanced to provide more value for patients. Second, the onboarding chatbot prompt may impact data quality and should be carefully considered. Finally, we highlighted several areas that could be improved by moving from a flow-based chatbot to a large language model implementation strategy.

Evaluating the WISE (Women in the South-East) Telehealth Network: A Model of Healthcare and Health Promotion at Rural Libraries

PURPOSE

The purpose of this study was to evaluate the effectiveness of the WISE (Women in the South-East) Telehealth Network.

DESIGN

A follow-up survey design was used to determine the impact of the program on access to healthcare.

SETTING

WISE provided preventive care to women and gender expansive people at local libraries and the Mobile Library in the rural South Carolina Lowcountry.

SUBJECTS

In 1 year (February 2021-2022), WISE reached 523 individuals with 151 agreeing to participate in the study. Most participants identified as white (66%) or Black (22%).

INTERVENTION

A Community Health Worker provided health education, connection to telehealth services, referrals, and connected individuals with community and social services.

MEASURES

The Telehealth Usability Questionnaire (TUQ), changes in knowledge, satisfaction with WISE, Acceptability of Intervention measure (AIM), and sociodemographic characteristics.

RESULTS

Participants with a high telehealth usability score were significantly more likely to be under the age of 35 (OR 4.60 [95% CI 1.21-17.52]), married (OR 10.00 [95% CI 2.19-45.64]), or white (OR 4.00 [95% CI 1.06-15.08]). The intervention earned a high acceptability score 4.46 (± .61)/5.0 by helping participants obtain necessary medical care and resources, as well as meeting their educational needs.

CONCLUSION

This study offers practical suggestions to expand the use of telehealth initiatives to improve health outcomes by engaging libraries in rural communities.

Roles, Users, Benefits, and Limitations of Chatbots in Health Care: Rapid Review

BACKGROUND

Chatbots, or conversational agents, have emerged as significant tools in health care, driven by advancements in artificial intelligence and digital technology. These programs are designed to simulate human conversations, addressing various health care needs. However, no comprehensive synthesis of health care chatbots' roles, users, benefits, and limitations is available to inform future research and application in the field.

OBJECTIVE

This review aims to describe health care chatbots' characteristics, focusing on their diverse roles in the health care pathway, user groups, benefits, and limitations.

METHODS

A rapid review of published literature from 2017 to 2023 was performed with a search strategy developed in collaboration with a health sciences librarian and implemented in the MEDLINE and Embase databases. Primary research studies reporting on chatbot roles or benefits in health care were included. Two reviewers dual-screened the search results. Extracted data on chatbot roles, users, benefits, and limitations were subjected to content analysis.

RESULTS

The review categorized chatbot roles into 2 themes: delivery of remote health services, including patient support, care management, education, skills building, and health behavior promotion, and provision of administrative assistance to health care providers. User groups spanned across patients with chronic conditions as well as patients with cancer; individuals focused on lifestyle improvements; and various demographic groups such as women, families, and older adults. Professionals and students in health care also emerged as significant users, alongside groups seeking mental health support, behavioral change, and educational enhancement. The benefits of health care chatbots were also classified into 2 themes: improvement of health care quality and efficiency and cost-effectiveness in health care delivery. The identified limitations encompassed ethical challenges, medicolegal and safety concerns, technical difficulties, user experience issues, and societal and economic impacts.

CONCLUSIONS

Health care chatbots offer a wide spectrum of applications, potentially impacting various aspects of health care. While they are promising tools for improving health care efficiency and quality, their integration into the health care system must be approached with consideration of their limitations to ensure optimal, safe, and equitable use.

User Preferences and Needs for Health Data Collection Using Research Electronic Data Capture: Survey Study

BACKGROUND

Self-administered web-based questionnaires are widely used to collect health data from patients and clinical research participants. REDCap (Research Electronic Data Capture; Vanderbilt University) is a global, secure web application for building and managing electronic data capture. Unfortunately, stakeholder needs and preferences of electronic data collection via REDCap have rarely been studied.

OBJECTIVE

This study aims to survey REDCap researchers and administrators to assess their experience with REDCap, especially their perspectives on the advantages, challenges, and suggestions for the enhancement of REDCap as a data collection tool.

METHODS

We conducted a web-based survey with representatives of REDCap member organizations in the United States. The survey captured information on respondent demographics, quality of patient-reported data collected via REDCap, patient experience of data collection with REDCap, and open-ended questions focusing on the advantages, challenges, and suggestions to enhance REDCap's data collection experience. Descriptive and inferential analysis measures were used to analyze quantitative data. Thematic analysis was used to analyze open-ended responses focusing on the advantages, disadvantages, and enhancements in data collection experience.

RESULTS

A total of 207 respondents completed the survey. Respondents strongly agreed or agreed that the data collected via REDCap are accurate (188/207, 90.8%), reliable (182/207, 87.9%), and complete (166/207, 80.2%). More than half of respondents strongly agreed or agreed that patients find REDCap easy to use (165/207, 79.7%), could successfully complete tasks without help (151/207, 72.9%), and could do so in a timely manner (163/207, 78.7%). Thematic analysis of open-ended responses yielded 8 major themes: survey development, user experience, survey distribution, survey results, training and support, technology, security, and platform features. The user experience category included more than half of the advantage codes (307/594, 51.7% of codes); meanwhile, respondents reported higher challenges in survey development (169/516, 32.8% of codes), also suggesting the highest enhancement suggestions for the category (162/439, 36.9% of codes).

CONCLUSIONS

Respondents indicated that REDCap is a valued, low-cost, secure resource for clinical research data collection. REDCap's data collection experience was generally positive among clinical research and care staff members and patients. However, with the advancements in data collection technologies and the availability of modern, intuitive, and mobile-friendly data collection interfaces, there is a critical opportunity to enhance the REDCap experience to meet the needs of researchers and patients.

Potential of AI-Driven Chatbots in Urology: Revolutionizing Patient Care Through Artificial Intelligence

PURPOSE OF REVIEW

Artificial intelligence (AI) chatbots have emerged as a potential tool to transform urology by improving patient care and physician efficiency. With an emphasis on their potential advantages and drawbacks, this literature review offers a thorough assessment of the state of AI-driven chatbots in urology today.

RECENT FINDINGS

The capacity of AI-driven chatbots in urology to give patients individualized and timely medical advice is one of its key advantages. Chatbots can help patients prioritize their symptoms and give advice on the best course of treatment. By automating administrative duties and offering clinical decision support, chatbots can also help healthcare providers. Before chatbots are widely used in urology, there are a few issues that need to be resolved. The precision of chatbot diagnoses and recommendations might be impacted by technical constraints like system errors and flaws. Additionally, issues regarding the security and privacy of patient data must be resolved, and chatbots must adhere to all applicable laws. Important issues that must be addressed include accuracy and dependability because any mistakes or inaccuracies could seriously harm patients. The final obstacle is resistance from patients and healthcare professionals who are hesitant to use new technology or who value in-person encounters. AI-driven chatbots have the potential to significantly improve urology care and efficiency. However, it is essential to thoroughly test and ensure the accuracy of chatbots, address privacy and security concerns, and design user-friendly chatbots that can integrate into existing workflows. By exploring various scenarios and examining the current literature, this review provides an analysis of the prospects and limitations of implementing chatbots in urology.

Decentralized research technology use in multicenter clinical research studies based at U.S. academic research centers

Introduction

During the COVID-19 pandemic, research organizations accelerated adoption of technologies that enable remote participation. Now, there's a pressing need to evaluate current decentralization practices and develop appropriate research, education, and operations infrastructure. The purpose of this study was to examine current adoption of decentralization technologies in a sample of clinical research studies conducted by academic research organizations (AROs).

Methods

The setting was three data coordinating centers in the U.S. These centers initiated coordination of 44 clinical research studies during or after 2020, with national recruitment and enrollment, and entailing coordination between one and one hundred sites. We determined the decentralization technologies used in these studies.

Results

We obtained data for 44/44 (100%) trials coordinated by the three centers. Three technologies have been adopted across nearly all studies (98-100%): eIRB, eSource, and Clinical Trial Management Systems. Commonly used technologies included e-Signature (32/44, 73%), Online Payments Portals (26/44, 59%), ePROs (23/44, 53%), Interactive Response Technology (22/44, 50%), Telemedicine (19/44, 43%), and eConsent (18/44, 41%). Wearables (7/44,16%) and Online Recruitment Portals (5/44,11%) were less common. Rarely utilized technologies included Direct-to-Patient Portals (1/44, 2%) and Home Health Nurse Portals (1/44, 2%).

Conclusions

All studies incorporated some type of decentralization technology, with more extensive adoption than found in previous research. However, adoption may be strongly influenced by institution-specific IT and informatics infrastructure and support. There are inherent needs, responsibilities, and challenges when incorporating decentralization technology into a research study, and AROs must ensure that infrastructure and informatics staff are adequate.

Older adults' experience with virtual conversational agents for health data collection

Introduction

Virtual conversational agents (i.e., chatbots) are an intuitive form of data collection. Understanding older adults' experiences with chatbots could help identify their usability needs. This quality improvement study evaluated older adults' experiences with a chatbot for health data collection. A secondary goal was to understand how perceptions differed based on length of chatbot forms.

Methods

After a demographic survey, participants (≥60 years) completed either a short (21 questions), moderate (30 questions), or long (66 questions) chatbot form. Perceived ease-of-use, usefulness, usability, likelihood to recommend, and cognitive load were measured post-test. Qualitative and quantitative analyses were used.

Results

A total of 260 participants reported on usability and satisfaction metrics including perceived ease-of-use (5.8/7), usefulness (4.7/7), usability (5.4/7), and likelihood to recommend (Net Promoter Score = 0). Cognitive load (12.3/100) was low. There was a statistically significant difference in perceived usefulness between groups, with a significantly higher mean perceived usefulness for Group 1 than Group 3. No other group differences were observed. The chatbot was perceived as quick, easy, and pleasant with concerns about technical issues, privacy, and security. Participants provided suggestions to enhance progress tracking, edit responses, improve readability, and have options to ask questions.

Discussion

Older adults found the chatbot to be easy, useful, and usable. The chatbot required low cognitive load demonstrating it could be an enjoyable health data collection tool for older adults. These results will inform the development of a health data collection chatbot technology.