The Role of Sensory Changes in Everyday Technology use by People with Mild to Moderate Dementia

Positive Emotional Responses to Socially Assistive Robots in People With Dementia: Pilot Study

Background

Interventions and care that can evoke positive emotions and reduce apathy or agitation are important for people with dementia. In recent years, socially assistive robots used for better dementia care have been found to be feasible. However, the immediate responses of people with dementia when they are given multiple sensory modalities from socially assistive robots have not yet been sufficiently elucidated.

Objective

This study aimed to quantitatively examine the immediate emotional responses of people with dementia to stimuli presented by socially assistive robots using facial expression analysis in order to determine whether they elicited positive emotions.

Methods

This pilot study adopted a single-arm interventional design. Socially assistive robots were presented to nursing home residents in a three-step procedure: (1) the robot was placed in front of participants (visual stimulus), (2) the robot was manipulated to produce sound (visual and auditory stimuli), and (3) participants held the robot in their hands (visual, auditory, and tactile stimuli). Expression intensity values for "happy," "sad," "angry," "surprised," "scared," and "disgusted" were calculated continuously using facial expression analysis with FaceReader. Additionally, self-reported feelings were assessed using a 5-point Likert scale. In addition to the comparison between the subjective and objective emotional assessments, expression intensity values were compared across the aforementioned 3 stimuli patterns within each session. Finally, the expression intensity value for "happy" was compared between the different types of robots.

Results

A total of 29 participants (mean age 88.7, SD 6.2 years; n=27 female; Japanese version of Mini-Mental State Examination mean score 18.2, SD 5.1) were recruited. The expression intensity value for "happy" was the largest in both the subjective and objective assessments and increased significantly when all sensory modalities (visual, auditory, and tactile) were presented (median expression intensity 0.21, IQR 0.09-0.35) compared to the other 2 patterns (visual alone: median expression intensity 0.10, IQR 0.03-0.22; P<.001; visual and auditory: median expression intensity 0.10, IQR 0.04-0.23; P<.001). The comparison of different types of robots revealed a significant increase when all stimuli were presented by doll-type and animal-type robots, but not humanoid-type robots.

Conclusions

By quantifying the emotional responses of people with dementia, this study highlighted that socially assistive robots may be more effective in eliciting positive emotions when multiple sensory stimuli, including tactile stimuli, are involved. More studies, including randomized controlled trials, are required to further explore the effectiveness of using socially assistive robots in dementia care.

Social connectedness and the role of virtual reality: experiences and perceptions of people living with dementia and their caregivers

PURPOSE

People living with dementia are often at increased risk of becoming socially disconnected due to dementia-related challenges. In recent years, digital technology has been designed to help address the social health of people living with dementia and provide opportunities to promote or maintain their social connectedness. This paper presents the findings from phase two of a participatory action research project, which explored people living with dementia and their caregiver's experiences and perceptions of social connectedness and the potential role of Virtual Reality (VR) in promoting or maintaining same.

MATERIALS AND METHODS

People living with dementia (n = 8) and their informal caregivers (n = 8) participated in an individual, 1:1 online interview. Data analysis was guided by reflexive thematic analysis.

RESULTS

The findings presented four themes: social connectedness: lived experiences and insights, facilitating social connectedness, barriers to social connectedness and the potential of multi-user VR for social connectedness. People living with dementia experienced a range of personal, community and societal connectedness. Facilitators of social connectedness included supportive, non-judgemental, and reciprocal relationships, technology adoption, and personal and contextual facilitators. Dementia-related difficulties and periods of disruption or change were considered barriers to social connectedness. Multi-user VR was perceived as useful for promoting and maintaining social connectedness.

CONCLUSIONS

The perceived usefulness of multi-user VR for social connectedness indicates its potential for use with this population. Understanding the lived experiences, barriers, and facilitators of social connectedness will assist researchers and the human-computer interaction community to inform the design of future multi-user VR for social connectedness outcomes with people living with dementia and their caregivers.

Technology for activity participation in older people with mild cognitive impairment or dementia: expert perspectives and a scoping review

PURPOSE

This two-phased study aimed to collate, summarize and characterize - through the lens of an occupation-based, person-centred framework - ongoing research and practice featuring activity participation-supportive digital health technology (DHT) for direct use by older persons with mild cognitive impairment or Alzheimer's disease and related dementias (PwMCI/ADRD).

MATERIALS AND METHODS

Phase 1: Using scoping review procedures, PubMed, MEDLINE and PsycInfo were searched to identify primary research studies. Phase 2: Semi-structured interviews were completed with MCI/ADRD expert stakeholders identified through publicly available biographies and snowball referral. Thematic analysis was used to identify, synthesize and cross-compare emergent themes from both data sources that were subsequently organized into core facets of the Human Activity Assistive Technology (HAAT) model.

RESULTS

The scoping review resulted in 28 studies, which were primarily feasibility work with small sample sizes. Interviewed experts (N = 17) had 4+ years of MCI/ADRD experience, came from a variety of settings, and held myriad roles. Real world and research-based use of DHTs held some commonalities, particularly around support for social participation and instrumental activities of daily engagement. No DHT for sleep or work/volunteerism were noted in either phase. People with milder MCI/ADRD conditions were most often targeted users. Soft technology strategies facilitating implementation centred on product design (e.g., prompting software, customisability, multimedia/multisensory experiences), instructional methods and technology partner involvement.

CONCLUSIONS

This study demonstrates that although DHT supportive of activity participation is being studied and integrated into the lives of PwMCI/ADRD, there are still key opportunities for growth to meet the needs of diverse MCI/ADRD end users.Implications for rehabilitationMainstream digital health technologies (DHTs) are being utilized by persons with mild cognitive impairment and Alzheimer's disease and related dementias (PwMCI/ADRD) in everyday life, in limited capacities, to support social participation, leisure, health management and instrumental activities of daily living (IADL).Innovative research-based technologies to be used directly by PwMCI/ADRD are under development, particularly to facilitate management of ADL, social participation and IADL in persons with mild-to-moderate forms of cognitive impairment.Soft technology strategies to support technology implementation with MCI/ADRD target users include close attention to design of the technology (e.g., customisability, sensory stimulators and prompting features), instructional strategies that promote learning and motivation and involvement of technology partners to facilitate engagement with the technology.Future studies will require more robust research designs with transparent reports of participant characteristics and facilitative instructional methods to expand DHT's potential to account for and better meet the needs of diverse MCI/ADRD communities in real-world contexts.

Enriching Social Sharing for the Dementia Community: Insights from In-person and Online Social Programs

The dementia community faces major challenges in social engagements, which have been further complicated by the prolonged physical distancing measures due to the COVID-19 pandemic. Designing digital tools for in-person social sharing in family and care facility settings has been well explored, but comparatively little HCI work has focused on the design of community-based social technologies for virtual settings. We present our virtual fieldwork on remote social activities explored by one dementia community in response to the impacts of the pandemic. Building upon our previously published on-site fieldwork in this community, we expand on our initial publication by follow-up interviewing caregivers and facilitators and reflecting on a virtual social program. Through thematic analysis and contrasting in-person and online formats of the program, we deepened the understanding of virtual social engagements of the dementia community, examining their efforts to leverage physical objects and environments, enhance open and flexible experiences, and expand collaborative space. We propose to open new design opportunities through holistic approaches, including reimagining community social spaces, rethinking agency in people with dementia and caregivers, and diversifying HCI support across communities and stakeholders.

Mobile Phone Use by People with Mild to Moderate Dementia: Uncovering Challenges and Identifying Opportunities: Mobile Phone Use by People with Mild to Moderate Dementia

With the rising usage of mobile phones by people with mild dementia, and the documented barriers to technology use that exist for people with dementia, there is an open opportunity to study the specifics of mobile phone use by people with dementia. In this work we provide a first step towards filling this gap through an interview study with fourteen people with mild to moderate dementia. Our analysis yields insights into mobile phone use by people with mild to moderate dementia, challenges they experience with mobile phone use, and their ideas to address these challenges. Based on these findings, we discuss design opportunities to help achieve more accessible and supportive technology use for people with dementia. Our work opens up new opportunities for the design of systems focused on augmenting and enhancing the abilities of people with dementia.

Barriers to Online Dementia Information and Mitigation

There is growing interest in HCI to study ways to support access to accurate, accessible, relevant online health information for different populations. Yet, there remains a need to understand the barriers that are posed by the way our platforms are designed as well as how we might overcome these barriers for people with dementia. To address this, we conducted sixteen interviews and observation sessions with people with mild to moderate dementia. Our analysis uncovered four barriers to online health information and corresponding mitigation strategies that participants employed. We discuss how HCI researchers may apply these findings towards new technical approaches and standards concerning information accessibility and credibility for neurodiverse populations. Finally, we broaden the scope of HCI research to include investigations of the accessibility and credibility of online information for people with age-related cognitive impairment independent of proxies.

Understanding How Sensory Changes Experienced by Individuals with a Range of Age-Related Cognitive Changes can Effect Technology Use

Clinical researchers have identified sensory changes people with age-related cognitive changes, such as dementia and mild cognitive impairment, experience which are different from typical age-related sensory changes. Technology designers and researchers do not yet have an understanding of how these unique sensory changes affect technology use. This work begins to bridge the gap between the clinical knowledge of sensory changes and technology research and design through interviews with people with mild to moderate dementia, mild cognitive impairment, subjective cognitive decline, and healthcare professionals. This extended version of our ASSETS conference paper includes people with a range of age-related cognitive changes describing changes in vision, hearing, speech, dexterity, proprioception, and smell. We discuss each of these sensory changes and ways to leverage optimal modes of sensory interaction for accessible technology use with existing and emerging technologies. Finally, we discuss how accessible sensory stimulation may change across the spectrum of age-related cognitive changes.

Investigating the Potential of Artificial Intelligence Powered Interfaces to Support Different Types of Memory for People with Dementia

There has been a growing interest in HCI to understand the specific technological needs of people with dementia and supporting them in self-managing daily activities. One of the most difficult challenges to address is supporting the fluctuating accessibility needs of people with dementia, which vary with the specific type of dementia and the progression of the condition. Researchers have identified auto-personalized interfaces, and more recently, Artificial Intelligence or AI-driven personalization as a potential solution to making commercial technology accessible in a scalable manner for users with fluctuating ability. However, there is a lack of understanding on the perceptions of people with dementia around AI as an aid to their everyday technology use and its role in their overall self-management systems, which include other non-AI technology, and human assistance. In this paper, we present future directions for the design of AI-based systems to personalize an interface for dementia-related changes in different types of memory, along with expectations for AI interactions with the user with dementia.

Investigating Best Practices for Remote Summative Usability Testing with People with Mild to Moderate Dementia

People with dementia may miss out on the benefits of using technology, because they often find it difficult to use. Usability testing is one method to identify barriers and areas for improvement in technology. Unfortunately, usability testing is often not conducted with people with dementia, independent of their caregivers. Difficulty recruiting local participants with dementia who regularly use technology further compounds the problem. Remote methods have been proposed as one approach to recruiting hard-to-reach populations. Currently, it is unclear how to effectively conduct remote summative usability testing with people with dementia. We recruited 15 participants. Five took part in the pilot study and 10 participated in the main study. We identify best practices and make suggestions for remote summative usability tests with people who have mild to moderate dementia, independent of caregivers. We discuss our findings in three sections: (1) logistics for planning remote summative usability testing, (2) approaches for conducting remote summative usability testing, including modifications of research methods, and (3) considerations when evaluating findings from remote summative usability sessions. We also present modified usability testing methods we developed to meet the unique needs of users with mild to moderate dementia, and summarize lessons learned and new directions for research on this topic.