Investigating the Accessibility of Voice Assistants With Impaired Users: Mixed Methods Study

Use of Artificial Intelligence Techniques to Assist Individuals with Physical Disabilities

Assistive technologies (AT) enable people with disabilities to perform activities of daily living more independently, have greater access to community and healthcare services, and be more productive performing educational and/or employment tasks. Integrating artificial intelligence (AI) with various agents, including electronics, robotics, and software, has revolutionized AT, resulting in groundbreaking technologies such as mind-controlled exoskeletons, bionic limbs, intelligent wheelchairs, and smart home assistants. This article provides a review of various AI techniques that have helped those with physical disabilities, including brain-computer interfaces, computer vision, natural language processing, and human-computer interaction. The current challenges and future directions for AI-powered advanced technologies are also addressed.

Smart Speakers as an Environmental Control Unit for Severe Motor Dependence: The Case of a Young Adult with Duchenne Muscular Dystrophy

Duchenne muscular dystrophy (DMD) is a disease that primarily affects males and causes a gradual loss of muscle strength. This results in a deterioration of motor skills and functional mobility, which can impact the performance of various occupations. Individuals with DMD often rely heavily on caregivers to assist with daily activities, which can lead to caregiver burden. A case study was conducted to explore and describe potential variations in the performance of a young adult diagnosed with DMD and his caregivers resulting from the integration of smart speakers (SS)-controlled Internet of Things (IoT) devices in the home environment. The study also examined the potential of SS as an environment control unit (ECU) and analysed variations in caregiver burden. Smart devices and SS were installed in the most frequently used spaces, namely, the bedroom and living room. The study employed WebQDA software to perform content analysis and Microsoft Excel to calculate the scores of the structured instruments. The implementation of the IoT-assisted environment compensated for previously physical tasks, resulting in a slight increase in independent performance and reduced demands on caregivers.

You have interrupted me again!: making voice assistants more dementia-friendly with incremental clarification

In spontaneous conversation, speakers seldom have a full plan of what they are going to say in advance: they need to conceptualise and plan incrementally as they articulate each word in turn. This often leads to long pauses mid-utterance. Listeners either wait out the pause, offer a possible completion, or respond with an incremental clarification request (iCR), intended to recover the rest of the truncated turn. The ability to generate iCRs in response to pauses is therefore important in building natural and robust everyday voice assistants (EVA) such as Amazon Alexa. This becomes crucial with people with dementia (PwDs) as a target user group since they are known to pause longer and more frequently, with current state-of-the-art EVAs interrupting them prematurely, leading to frustration and breakdown of the interaction. In this article, we first use two existing corpora of truncated utterances to establish the generation of clarification requests as an effective strategy for recovering from interruptions. We then proceed to report on, analyse, and release SLUICE-CR: a new corpus of 3,000 crowdsourced, human-produced iCRs, the first of its kind. We use this corpus to probe the incremental processing capability of a number of state-of-the-art large language models (LLMs) by evaluating (1) the quality of the model's generated iCRs in response to incomplete questions and (2) the ability of the said LLMs to respond correctly after the users response to the generated iCR. For (1), our experiments show that the ability to generate contextually appropriate iCRs only emerges at larger LLM sizes and only when prompted with example iCRs from our corpus. For (2), our results are in line with (1), that is, that larger LLMs interpret incremental clarificational exchanges more effectively. Overall, our results indicate that autoregressive language models (LMs) are, in principle, able to both understand and generate language incrementally and that LLMs can be configured to handle speech phenomena more commonly produced by PwDs, mitigating frustration with today's EVAs by improving their accessibility.

Efficacy of Alexa, Google Assistant, and Siri for Supporting Informed Prostate Cancer Screening Decisions for African-American Men

Prostate cancer is the most prevalent non-skin cancer among all men, but African-Americans have morbidity and mortality at significantly higher rates than White men. To reduce this burden, authorities such as the American Cancer Society recommend that men make informed/shared screening decisions with a healthcare provider. Informed/shared screening decisions require that men have adequate prostate cancer knowledge. Virtual assistants are interactive communication technologies that have become popular for seeking health information, though information quality has been mixed. No prior research has investigated the quality of prostate cancer information disseminated by virtual assistants. The purpose of this study was to determine the response rates, accuracy, breadth, and credibility of three popular virtual assistants (Alexa, Google Assistant, and Siri) for supporting informed/shared prostate cancer screening decisions for African-American men. Each virtual assistant was evaluated on a tablet, cell phone, and smart speaker using 12 frequently asked screening questions. Responses were rated dichotomously (i.e., yes/no), and analyses were conducted using SPSS. Alexa on a phone or tablet and Google Assistant on a smart speaker had the best overall performance based on a combination of response, accuracy, and credibility scores. All other assistants scored below 75% in one or more areas. Additionally, all virtual assistants lacked the breadth to support an informed/shared prostate cancer screening decision. African-American men may be especially disadvantaged by using virtual assistants for prostate cancer information because of the lack of emphasis on their greater disease risk, higher mortality rates, and appropriate ages at which they should begin screening conversations.

A Survey of Conversational Agents and Their Applications for Self-Management of Chronic Conditions

Conversational agents have gained their ground in our daily life and various domains including healthcare. Chronic condition self-management is one of the promising healthcare areas in which conversational agents demonstrate significant potential to contribute to alleviating healthcare burdens from chronic conditions. This survey paper introduces and outlines types of conversational agents, their generic architecture and workflow, the implemented technologies, and their application to chronic condition self-management.

A Method for Improving the Prediction of Outpatient Visits for Hospital Management: Bayesian Autoregressive Analysis

The number of outpatient visits is generally influenced by various factors that are difficult to quantify and obtain, resulting in some irregular fluctuations. The traditional statistical methodology seldom considers these uncertainties. Accordingly, this paper presents a Bayesian autoregressive (AR) analysis to propose a forecasting framework to cope with the strict requirements. The AR model was conducted to identify the linear and autocorrelation relationships of historical series, and Bayesian inference was used to correct and optimize the AR model parameters. Posterior distribution of parameters was stably and reliably obtained by Gibbs sampling on the condition of the convergent Markov chain. Meanwhile, the lag orders of the AR model were adjusted based on the series characteristics. To increase the variability and generality of the dataset, the developed Bayesian AR model was evaluated at seven hospitals in China. The results demonstrated that the Bayesian AR model had varying degrees of decline in the MAPE value in the seven sets of experimental data. The reductions ranged from 0.1431% to 0.0342%, indicating effective optimization of the Bayesian inference in the AR model parameters and reflecting the useful correction of the lag order adjustment strategy. The proposed Bayesian AR framework showed high accuracy index and stable prediction accuracy, thereby outperforming the traditional AR model.

Should Alexa diagnose Alzheimer's?: Legal and ethical issues with at-home consumer devices

Voice-based AI-powered digital assistants, such as Alexa, Siri, and Google Assistant, present an exciting opportunity to translate healthcare from the hospital to the home. But building a digital, medical panopticon can raise many legal and ethical challenges if not designed and implemented thoughtfully. This paper highlights the benefits and explores some of the challenges of using digital assistants to detect early signs of cognitive impairment, focusing on issues such as consent, bycatching, privacy, and regulatory oversight. By using a fictional but plausible near-future hypothetical, we demonstrate why an "ethics-by-design" approach is necessary for consumer-monitoring tools that may be used to identify health concerns for their users.

Using Voice and Touchscreen Controlled Smart Speakers to Protect Vulnerable Clients in Long Term Care Facilities

Background and Objectives

The Centers for Medicare and Medicaid Services (CMS) restricted long term care (LTC) facility visitation to only essential personnel during the COVID-19 pandemic. The Maryland Department of Human Services distributed Amazon Echoshow 8 voice and touchscreen controlled smart speakers (VTCSS) to a sample of their institutionalized guardianship clients to enhance caseworker access during the pandemic.

Research Design and Methods

This pilot study focused on understanding VTCSS use challenges and the effects on clients’ safety and well-being. Two focus groups were conducted with caseworkers (N=16) who piloted the devices. The interviews were recorded, transcribed, and analyzed using open and axial coding.

Results

Four themes were identified, including challenges to providing casework during the pandemic (e.g. facility technology gaps), challenges to device installation and use (e.g. privacy concerns), strategies for overcoming challenges (e.g. alert features), and benefits (e.g. stimulation, care monitoring) and uses (e.g. enhanced access, entertainment).

Discussion and Implications

VTCSS show great promise to engage the client, maintain visual access, and monitor quality of care. However, facilitating access to such technology requires planning and training before installation.

"It's Kind of Like Code-Switching": Black Older Adults' Experiences with a Voice Assistant for Health Information Seeking

Black older adults from lower socioeconomic environments are often neglected in health technology interventions. Voice assistants have a potential to make healthcare more accessible to older adults, yet, little is known about their experiences with this type of health information seeking, especially Black older adults. Through a three-phase exploratory study, we explored health information seeking with 30 Black older adults in lower-income environments to understand how they ask health-related questions, and their perceptions of the Google Home being used for that purpose. Through our analysis, we identified the health information needs and common search topics, and discussed the communication breakdowns and types of repair performed. We contribute an understanding of cultural code-switching that has to be done by these older adults when interacting with voice assistants, and the importance of such phenomenon when designing for historically excluded groups.

Speech and Language Practitioners' Experiences of Commercially Available Voice-Assisted Technology: Web-Based Survey Study

Background

Speech and language therapy involves the identification, assessment, and treatment of children and adults who have difficulties with communication, eating, drinking, and swallowing. Globally, pressing needs outstrip the availability of qualified practitioners who, of necessity, focus on individuals with advanced needs. The potential of voice-assisted technology (VAT) to assist people with speech impairments is an emerging area of research but empirical work exploring its professional adoption is limited.

Objective

This study aims to explore the professional experiences of speech and language therapists (SaLTs) using VAT with their clients to identify the potential applications and barriers to VAT adoption and thereby inform future directions of research.

Methods

A 23-question survey was distributed to the SaLTs from the United Kingdom using a web-based platform, eliciting both checkbox and free-text responses, to questions on perceptions and any use experiences of VAT. Data were analyzed descriptively with content analysis of free text, providing context to their specific experiences of using VAT in practice, including barriers and opportunities for future use.

Results

A total of 230 UK-based professionals fully completed the survey; most were technologically competent and were aware of commercial VATs (such as Alexa and Google Assistant). However, only 49 (21.3%) SaLTs had used VAT with their clients and described 57 use cases. They reported using VAT with 10 different client groups, such as people with dysarthria and users of augmentative and alternative communication technologies. Of these, almost half (28/57, 49%) used the technology to assist their clients with day-to-day tasks, such as web browsing, setting up reminders, sending messages, and playing music. Many respondents (21/57, 37%) also reported using the technology to improve client speech, to facilitate speech practice at home, and to enhance articulation and volume. Most reported a positive impact of VAT use, stating improved independence (22/57, 39%), accessibility (6/57, 10%), and confidence (5/57, 8%). Some respondents reported increased client communication (5/57, 9%) and sociability (3/57, 5%). Reasons given for not using VAT in practice included lack of opportunity (131/181, 72.4%) and training (63/181, 34.8%). Most respondents (154/181, 85.1%) indicated that they would like to try VAT in the future, stating that it could have a positive impact on their clients’ speech, independence, and confidence.

Conclusions

VAT is used by some UK-based SaLTs to enable communication tasks at home with their clients. However, its wider adoption may be limited by a lack of professional opportunity. Looking forward, additional benefits are promised, as the data show a level of engagement, empowerment, and the possibility of achieving therapeutic outcomes in communication impairment. The disparate responses suggest that this area is ripe for the development of evidence-based clinical practice, starting with a clear definition, outcome measurement, and professional standardization.

Reliability of Commercial Voice Assistants' Responses to Health-Related Questions in Noncommunicable Disease Management: Factorial Experiment Assessing Response Rate and Source of Information

BACKGROUND

Noncommunicable diseases (NCDs) constitute a burden on public health. These are best controlled through self-management practices, such as self-information. Fostering patients' access to health-related information through efficient and accessible channels, such as commercial voice assistants (VAs), may support the patients' ability to make health-related decisions and manage their chronic conditions.

OBJECTIVE

This study aims to evaluate the reliability of the most common VAs (ie, Amazon Alexa, Apple Siri, and Google Assistant) in responding to questions about management of the main NCD.

METHODS

We generated health-related questions based on frequently asked questions from health organization, government, medical nonprofit, and other recognized health-related websites about conditions associated with Alzheimer's disease (AD), lung cancer (LCA), chronic obstructive pulmonary disease, diabetes mellitus (DM), cardiovascular disease, chronic kidney disease (CKD), and cerebrovascular accident (CVA). We then validated them with practicing medical specialists, selecting the 10 most frequent ones. Given the low average frequency of the AD-related questions, we excluded such questions. This resulted in a pool of 60 questions. We submitted the selected questions to VAs in a 3×3×6 fractional factorial design experiment with 3 developers (ie, Amazon, Apple, and Google), 3 modalities (ie, voice only, voice and display, display only), and 6 diseases. We assessed the rate of error-free voice responses and classified the web sources based on previous research (ie, expert, commercial, crowdsourced, or not stated).

RESULTS

Google showed the highest total response rate, followed by Amazon and Apple. Moreover, although Amazon and Apple showed a comparable response rate in both voice-and-display and voice-only modalities, Google showed a slightly higher response rate in voice only. The same pattern was observed for the rate of expert sources. When considering the response and expert source rate across diseases, we observed that although Google remained comparable, with a slight advantage for LCA and CKD, both Amazon and Apple showed the highest response rate for LCA. However, both Google and Apple showed most often expert sources for CVA, while Amazon did so for DM.

CONCLUSIONS

Google showed the highest response rate and the highest rate of expert sources, leading to the conclusion that Google Assistant would be the most reliable tool in responding to questions about NCD management. However, the rate of expert sources differed across diseases. We urge health organizations to collaborate with Google, Amazon, and Apple to allow their VAs to consistently provide reliable answers to health-related questions on NCD management across the different diseases.

Smart Co-housing for People With Disabilities: A Preliminary Assessment of Caregivers' Interaction With the DOMHO System

Millions of people with motor and cognitive disabilities face hardships in daily life due to the limited accessibility and inclusiveness of living spaces which limit their autonomy and independence. The DOMHO project deals with these fundamental issues by leveraging an innovative solution: a smart co-housing apartment. Besides, the project aims at exploiting the well know effects of co-housing on individuals' health and well-being in combination with ambient assisted living technologies. The present study focused on the interaction of caregivers with the control application of an integrated smart system. Participants performed different tasks, fill out a questionnaire, and were interviewed. Performance and usability of the user interface, trust in technology, privacy, and attitudes towards home automation were explored. A series of guidelines for domotic technology control interfaces design was identified, and a high level of trust in these advanced tools was shown. Caregivers considered smart technologies as a work aid and a means for enhancing autonomy and life quality for users with disabilities.

Voice-Based Conversational Agents for the Prevention and Management of Chronic and Mental Health Conditions: Systematic Literature Review

BACKGROUND

Chronic and mental health conditions are increasingly prevalent worldwide. As devices in our everyday lives offer more and more voice-based self-service, voice-based conversational agents (VCAs) have the potential to support the prevention and management of these conditions in a scalable manner. However, evidence on VCAs dedicated to the prevention and management of chronic and mental health conditions is unclear.

OBJECTIVE

This study provides a better understanding of the current methods used in the evaluation of health interventions for the prevention and management of chronic and mental health conditions delivered through VCAs.

METHODS

We conducted a systematic literature review using PubMed MEDLINE, Embase, PsycINFO, Scopus, and Web of Science databases. We included primary research involving the prevention or management of chronic or mental health conditions through a VCA and reporting an empirical evaluation of the system either in terms of system accuracy, technology acceptance, or both. A total of 2 independent reviewers conducted the screening and data extraction, and agreement between them was measured using Cohen kappa. A narrative approach was used to synthesize the selected records.

RESULTS

Of 7170 prescreened papers, 12 met the inclusion criteria. All studies were nonexperimental. The VCAs provided behavioral support (n=5), health monitoring services (n=3), or both (n=4). The interventions were delivered via smartphones (n=5), tablets (n=2), or smart speakers (n=3). In 2 cases, no device was specified. A total of 3 VCAs targeted cancer, whereas 2 VCAs targeted diabetes and heart failure. The other VCAs targeted hearing impairment, asthma, Parkinson disease, dementia, autism, intellectual disability, and depression. The majority of the studies (n=7) assessed technology acceptance, but only few studies (n=3) used validated instruments. Half of the studies (n=6) reported either performance measures on speech recognition or on the ability of VCAs to respond to health-related queries. Only a minority of the studies (n=2) reported behavioral measures or a measure of attitudes toward intervention-targeted health behavior. Moreover, only a minority of studies (n=4) reported controlling for participants' previous experience with technology. Finally, risk bias varied markedly.

CONCLUSIONS

The heterogeneity in the methods, the limited number of studies identified, and the high risk of bias show that research on VCAs for chronic and mental health conditions is still in its infancy. Although the results of system accuracy and technology acceptance are encouraging, there is still a need to establish more conclusive evidence on the efficacy of VCAs for the prevention and management of chronic and mental health conditions, both in absolute terms and in comparison with standard health care.