A Virtual Therapist for Speech and Language Therapy

Embracing virtual reality in rehabilitation of post-stroke aphasia

About one-third of stroke survivors experience aphasia, i.e., language dysfunction caused by brain damage. Aphasia affects not only a person's ability to communicate, but it often leads to the inability to return to work, loss of close relationships, diminished quality of life, negative self-perception, and depression. Yet persons with aphasia are globally underserved due to limited access to resources, which limits their chance for recovery. Immersive virtual reality (VR) has the potential to solve this problem and deliver efficient, personalized treatments to millions of people worldwide who need access to rehabilitation services or more flexibility in treatment delivery. To reduce the global burden of stroke experts recommend taking bold, pragmatic actions across all four pillars of stroke quadrangle-surveillance, prevention, acute care, and rehabilitation. Embracing immersive VR-based rehabilitation of poststroke aphasia would be one step in that direction.

Digital games for rehabilitation of speech disorders: A scoping review

Background and Aims

Digital games are among the treatment methods for speech disorders that serve purposes other than mere entertainment. These games have been used for different speech disorders at any age. This study aims to review articles that have used digital games for rehabilitating speech disorders.

Methods

This study was a scoping review. PubMed, Scopus, and Web of Science were searched on February 28, 2022, to access the articles on digital games used in rehabilitation of speech disorders without any date restrictions. The search strategy was as follows: ("video game [MeSH term]" OR "computer game" OR "mobile game" OR "serious game" OR gamification [MeSH term]) AND ("speech pathology" OR "speech therapy [MeSH term]" OR "speech disorder [MeSH term]" OR stuttering [MeSH term]). Original interventional and observational studies in English were included. The data were extracted from the relevant articles, including the first author's name, year of publication, country, target group, participants, mobile device/computer-based, type of game design, language level, number of sessions, and outcome. Descriptive statistics were used to analyze the data.

Results

Of 693 retrieved articles, 10 articles were included in this study. Digital games were used for different speech disorders such as apraxia (20%), dysarthria (10%), articulatory hypokinesia in Parkinson's disease (10%), dysphonic disorder (10%), hearing disability (10%), phonological impairment (10%), and speech disorder in autism (10%). Most of the articles (60%) used a mobile device-based game. Phonemes (30%), words (30%), and sentences (20%) were the most frequently used language levels in designing digital games. All the reviewed articles reported the positive effect of digital games on speech and the patients' motivation in therapy.

Conclusion

Digital games can improve patients' speech and motivation in therapy. Although studies showed the positive impact of digital games on speech disorders, personalized speech therapy should be considered in designing these games.

Automatic boost articulation therapy in adults with dysarthria: Acceptability, usability and user interaction

BACKGROUND

Imprecise articulation has a negative impact on speech intelligibility. Therefore, treatment of articulation is clinically relevant in patients with dysarthria. In order to be effective and according to the principles of motor learning, articulation therapy needs to be intensive, well organized, with adequate feedback and requires frequent practice.

AIMS

The aims of this pilot study are (1) to evaluate the feasibility of a virtual articulation therapy (VAT) to guide patients with dysarthria through a boost articulation therapy (BArT) program; (2) to evaluate the acoustic models' performance used for automatic phonological error detection; and (3) to validate the system by end-users from their perspective.

METHODS & PROCEDURES

The VAT provides an extensive and well-structured package of exercises with visual and auditory modelling and adequate feedback on the utterances. The tool incorporates automated methods to detect phonological errors, which are specifically designed to analyse Dutch speech production. A total of 14 subjects with dysarthria evaluated the acceptability, usability and user interaction with the VAT based on two completed therapy sessions using a self-designed questionnaire.

OUTCOMES & RESULTS

In general, participants were positive about the new computer-based therapy approach. The algorithm performance for phonological error detection shows it to be accurate, which contributes to adequate feedback of utterance production. The results of the study indicate that the VAT has a user-friendly interface that can be used independently by patients with dysarthria who have sufficient cognitive, linguistic, motoric and sensory skills to benefit from speech therapy. Recommendations were given by the end-users to further optimize the program and to ensure user engagement.

CONCLUSIONS & IMPLICATIONS

The initial implementation of an automatic BArT shows it to be feasible and well accepted by end-users. The tool is an appropriate solution to increase the frequency and intensity of articulation training that supports traditional methods.

WHAT THIS PAPER ADDS

What is already known on the subject Behavioural interventions to improve articulation in patients with dysarthria demand intensive treatments, repetitive practice and feedback. However, the current treatments are mainly limited in time to the interactive sessions in the presence of speech-language pathology. Automatic systems addressing the needs of individuals with dysarthria are scarce. This study evaluates the feasibility of a VAT program and investigates its acceptability, usability and user interaction. What this paper adds to existing knowledge The computer-based speech therapy approach developed and applied in this study intends to support intensive articulation training of patients with dysarthria. The virtual speech therapy offers the possibility of an individualized and customized therapy programme, with an extensive database of exercises, visual and auditory models of the target utterances, and providing adequate feedback based on automatic acoustic analysis of speech. What are the potential or actual clinical implications of this work? The automatic BArT overcomes the limitation in time of face-to-face traditional speech therapy. It offers patients the opportunity to have access to speech therapy more intensively and frequently in their home environment.

An overview of the digital solutions for helping people with aphasia through bibliometric analysis

This paper gives insights into recent research developments in the field of digital solutions for people with aphasia and tries to analyse its impact on their rehabilitation. A bibliometric research approach is used for data collection. Relevant studies were extracted from seven major academic databases from years 2000 to 2019 inclusive. The systematic process resulted in 986 studies. The average growth in this field is 4%, which is less compared to other research areas. However, the average citations per paper is 7.27, which represents a medium level of publication quality. Scopus and Web of Science are leading databases for the number of studies (379 and 264) and quality of publications (P-Index: 49.26 and 32.85), respectively. The USA, with 42% of publications, leads this research field, followed by the UK with 15%. Both countries have national aphasia strategies in place through charities (not government level strategies), which potentially contributed to their research leadership. The results show that recent advances in digital solutions have the potential to support people with aphasia. However, further work is needed at both academic and government levels to deliver more research contributions and funding for the rehabilitation of people with aphasia.

•

The countries with clear national aphasia strategies in place lead this research field.

•

Western countries carry most of the research for aphasia; the global view of this research area is still messing.

•

Personal care, in combination with digital solutions, can yield better results.

•

The fully immersive virtual reality solutions offer close to real-life interaction experience for the people with aphasia and their surroundings inside the virtual space.

Usability study of wearable inertial sensors for exergames (WISE) for movement assessment and exercise

BACKGROUND

Accurate assessment of movement limitations and compliance monitoring of exercises to restore movement are necessary to tailor treatments for individuals with motor deficits. Although several commercial-grade technologies are available to clinicians for evaluating movement limitations, they require one-on-one time-consuming assessments with limited reproducibility across care settings. To address these limitations, a wearable inertial sensors for exergames (WISE) system has been designed with: (I) an animated virtual coach to deliver instruction and (II) a subject-model whose movements are animated by real-time sensor measurements from the WISE system worn by a subject. This paper examines the WISE system's accuracy and usability for the assessment of upper limb range of motion (ROM).

METHODS

Seventeen neurologically intact subjects were recruited to participate in a usability study of the WISE system. The subjects performed five shoulder and elbow exercises for each arm instructed by the animated virtual coach. The accuracy of ROM measurements obtained with the WISE system versus those obtained with the Kinect™ were compared using the root mean square error (RMSE) of the computed joint angles. The subjects additionally completed a system usability scale (SUS) to evaluate the usability of the virtual coach for tutoring ROM exercises.

RESULTS

The absolute agreement between the WISE and Kinect devices was moderate to very good and it was limited because the Kinect sensor suffers from occlusion. The Bland-Altman limits of agreement for the exercises in the coronal and transverse planes were within the acceptable limits of ±10°. The SUS response data produced relatively high third and first quartile scores of 97.5 and 82.5, respectively, with the interquartile range of 15 and the minimum score of 65, suggesting that the subjects were interested in using the animated virtual coach for tutoring ROM exercises.

CONCLUSIONS

An animated virtual coach-based WISE system for mHealth is presented, tested, and validated for guided upper limb ROM exercises. Future studies with patient populations will facilitate the use of these devices in clinical and telerehabilitation settings.

Can Script Training Improve Narrative and Conversation in Aphasia across Etiology?

Script training is an effective treatment of stable (e.g., stroke-induced) and progressive aphasia of varying severities and subtypes. The theoretical underpinnings of script training are discussed and include fluency-inducing conditions, speech shadowing, principles of neuroplasticity, and automatization. Script training outcomes are reviewed, with a focus on discourse in persons with stable aphasia (PWSAs) and in persons with primary progressive aphasia (PWPPAs). PWSAs and PWPPAs are able to acquire and maintain short scripted monologues or conversational dialogues, with some evidence of generalization to untrained topics and settings. Advances in both technology and access have enriched script training protocols, so they now range from no-tech written script approaches to high-tech audiovisual support and avatars. Advances in audio and/or visual support promote large amounts of practice of less errorful whole-message language processing during a fluent language inducing condition. With enough practice, users decrease reliance on supports and independently produce scripted content. Script training can be delivered in a variety of settings (individual, group, telepractice), lends itself well to homework programs, and is in accordance with the principles of neuroplasticity for neurorehabilitation. Incorporating script training into therapy programming is advantageous throughout aphasia recovery following brain injuries such as stroke. It is also beneficial for persons with progressive disease for prophylaxis, remediation, and compensation. Recommendations for implementing script training in clinical practice and future research directions are presented.

Rehabilitation, the Great Absentee of Virtual Coaching in Medical Care: Scoping Review

BACKGROUND

In the last few years, several studies have focused on describing and understanding how virtual coaches (ie, coaching program or smart device aiming to provide coaching support through a variety of application contexts) could be key drivers for health promotion in home care settings. As there has been enormous technological progress in the field of artificial intelligence and data processing in the past decade, the use of virtual coaches gains an augmented attention in the considerations of medical innovations.

OBJECTIVE

This scoping review aimed at providing an overview of the applications of a virtual coach in the clinical field. In particular, the review focused on the papers that provide tangible information for coaching activities with an active implication for engaging and guiding patients who have an ongoing plan of care.

METHODS

We aimed to investigate the use of the term virtual coach in the clinical field performing a methodical review of the relevant literature indexed on PubMed, Scopus, and Embase databases to find virtual coach papers focused on specific activities dealing with clinical or medical contexts, excluding those aimed at surgical settings or electronic learning purposes.

RESULTS

After a careful revision of the inclusion and exclusion criteria, 46 records were selected for the full-text review. Most of the identified articles directly or indirectly addressed the topic of physical activity. Some papers were focused on the use of virtual coaching (VC) to manage overweight or nutritional issues. Other papers dealt with technological interfaces to facilitate interactions with patients suffering from different chronic clinical conditions such as heart failure, chronic obstructive pulmonary disease, depression, and chronic pain.

CONCLUSIONS

Although physical activity is a healthy practice that is most encouraged by a virtual coach system, in the current scenario, rehabilitation is the great absentee. This paper gives an overview of the tangible applications of this tool in the medical field and may inspire new ideas for future research on VC.

Optimising recovery in aphasia: Learning following exposure to a single dose of computer-based script training

Purpose: Intensity of therapy influences outcomes in aphasia treatment, but little is known about optimum dosage across treatment approaches. Improved understanding of the effects of dose could help facilitate clinical decisions that maximise patient outcomes. This study examines learning in individuals with aphasia following exposure to one 60-min dose of computer-delivered script training, with or without a rest break. Method: Twenty participants (6F, 14M; 17 nonfluent, 3 fluent) with chronic aphasia due to left-hemisphere stroke participated. Participants had a mean (SD) age of 56.9 (8.4) years, a mean time post onset of 55.1 (range: 6.4-396.4) months, and a mean (SD) Comprehensive Aphasia Test score of 47.8 (4.0). Participants were exposed to 60 min of training, with or without a rest break, of a three-turn conversational script about either weather or traffic. Treatment was delivered via computer using a virtual clinician that visually modelled speech and guided treatment, promoting treatment fidelity. Oral reading probes for script sentences were administered at baseline (3 times), mid-treatment, immediately post-treatment, and, at various times, up to 2 weeks post-treatment to track script acquisition and maintenance. The study also examined generalisation from a trained script to a conversation with a clinician via a generalisation probe at three time points: baseline, immediately post-treatment, and 2 weeks post-treatment. Result: Following exposure to one dose of script training, participants demonstrated statistically significant improvements in oral reading accuracy and rate of script production on trained probes from baseline to immediately post-treatment and 2 weeks post-treatment. Participants also demonstrated significant change from baseline in generalisation to a conversation with a clinician immediately post-treatment and at 2 weeks post-treatment. Conclusion: Exposure to a single dose of computer-delivered script training may facilitate learning. These results have implications for future research investigating optimal dosage and scheduling parameters.

Using Health Chatbots for Behavior Change: A Mapping Study

This study conducts a mapping study to survey the landscape of health chatbots along three research questions: What illnesses are chatbots tackling? What patient competences are chatbots aimed at? Which chatbot technical enablers are of most interest in the health domain? We identify 30 articles related to health chatbots from 2014 to 2018. We analyze the selected articles qualitatively and extract a triplet <technicalEnablers, competence, illness> for each of them. This data serves to provide a first overview of chatbot-mediated behavior change on the health domain. Main insights include: nutritional disorders and neurological disorders as the main illness areas being tackled; "affect" as the human competence most pursued by chatbots to attain change behavior; and "personalization" and "consumability" as the most appreciated technical enablers. On the other hand, main limitations include lack of adherence to good practices to case-study reporting, and a deeper look at the broader sociological implications brought by this technology.

The Table to Tablet (T2T) Speech and Language Therapy Software Development Roadmap

Background

Few studies have analyzed gains in using computers in speech and language therapy interventions for children with speech and/or language disorders when compared to a control group, but virtual tutors and computer-based visual feedback have been gaining interest in the literature. Previous systematic reviews mainly focused on development technological details of computer-based speech training systems or the potential of integrating mobile technology into education and rehabilitation, but recent systematic reviews have also evaluated the efficacy of computer-based speech and language therapy for children and how digital technology can support different activities, at school or elsewhere.

Objective

This study aimed to analyze a continuous communication and joint team approach to develop solutions focused on the real needs of end users, which digitally emulate reliable and validated physical intervention materials for children with speech sound disorders (SSD).

Methods

The Table to Tablet (T2T) software was developed using a design-based research methodology, which included four phases: activities development; ethnographic pretesting with a sample from the target population; software development; and beta-testing. The technology used to develop the software, the method used to ensure satisfaction and replay ability of the intervention materials, and results from the ethnographic and beta-testing phases are presented.

Results

Nineteen activities were developed during the first phase, which were then tested, with 7 service users, using a physical prototype. The beta-test approach included extensive testing and reformulation, supported by direct, nonparticipant observation and data collection using a questionnaire designed for children. Feedback was used to improve the software and interaction with users.

Conclusions

The use of T2T-based intervention programmes by speech and language therapists (SLTs) will allow these professionals to make a better and more effective communication intervention, based on proven methodologies, that coexists in a structured physical and a digital version. These versions provide a full, 6-week intervention program, with minimal effort in preparing the session by the SLTs while delivering a very consistent intervention, with high replay value. A continuous communication and joint team approach was beneficial to the project and to the development of a solution focused on the real needs of SLTs and children with SSD. All problems were approached as a team with different skills and expertise, which minimized errors (eg, the developer making choices that would save him from spending time doing something that would not be used) and time spent. To add to this, the importance of integrating the end users as testers and collecting their opinions and actions per session allowed the production of better-targeted activities.

Trial Registration

ClinicalTrials.gov NCT02490826; https://clinicaltrials.gov/ct2/show/NCT02490826

Script Training Using Telepractice with Two Adults with Chronic Non-Fluent Aphasia

Two male participants with chronic (> 2 years), non-fluent aphasia and their family members participated in script training using videoconferencing. Functional scripts were developed by people with aphasia (PWA) and their family members. Accuracy of scripts was measured by total target words produced per turn. Participant 1 with Broca’s aphasia produced scripts with 0% accuracy pre-treatment and 87.5% accuracy post-treatment. Participant 2 with Transcortical Motor aphasia produced scripts with 20.2% accuracy pre-treatment and 63.5% accuracy post-treatment. Pre- and post-questionnaires for communication effectiveness and the use of telepractice for speech therapy indicated improvements in answering yes/no questions, participating in conversations with strangers, and increasing confidence and satisfaction with technology delivered treatment. The use of videoconferencing to deliver script training appears beneficial for individuals with chronic aphasia.

How a diverse research ecosystem has generated new rehabilitation technologies: Review of NIDILRR's Rehabilitation Engineering Research Centers

Over 50 million United States citizens (1 in 6 people in the US) have a developmental, acquired, or degenerative disability. The average US citizen can expect to live 20% of his or her life with a disability. Rehabilitation technologies play a major role in improving the quality of life for people with a disability, yet widespread and highly challenging needs remain. Within the US, a major effort aimed at the creation and evaluation of rehabilitation technology has been the Rehabilitation Engineering Research Centers (RERCs) sponsored by the National Institute on Disability, Independent Living, and Rehabilitation Research. As envisioned at their conception by a panel of the National Academy of Science in 1970, these centers were intended to take a "total approach to rehabilitation", combining medicine, engineering, and related science, to improve the quality of life of individuals with a disability. Here, we review the scope, achievements, and ongoing projects of an unbiased sample of 19 currently active or recently terminated RERCs. Specifically, for each center, we briefly explain the needs it targets, summarize key historical advances, identify emerging innovations, and consider future directions. Our assessment from this review is that the RERC program indeed involves a multidisciplinary approach, with 36 professional fields involved, although 70% of research and development staff are in engineering fields, 23% in clinical fields, and only 7% in basic science fields; significantly, 11% of the professional staff have a disability related to their research. We observe that the RERC program has substantially diversified the scope of its work since the 1970's, addressing more types of disabilities using more technologies, and, in particular, often now focusing on information technologies. RERC work also now often views users as integrated into an interdependent society through technologies that both people with and without disabilities co-use (such as the internet, wireless communication, and architecture). In addition, RERC research has evolved to view users as able at improving outcomes through learning, exercise, and plasticity (rather than being static), which can be optimally timed. We provide examples of rehabilitation technology innovation produced by the RERCs that illustrate this increasingly diversifying scope and evolving perspective. We conclude by discussing growth opportunities and possible future directions of the RERC program.

Experiencing EVA Park, a Multi-User Virtual World for People with Aphasia

Virtual worlds are used in wide-ranging ways by many people with long-term health conditions, but their use by people with aphasia (PWA) has been limited. In contrast, this article reports the use of EVA Park, a multi-user virtual world designed for PWA to practice conversations, focusing on people's emotional, social, and conversational experiences. An analysis of observation and interview data collected from 20 people with aphasia who participated in a 5-week therapy intervention revealed key themes related to user experience. The themes offer a rich insight into aspects of the virtual world experience for PWA that go beyond therapeutic outcomes. They are as follows: affect (positive and negative); types of conversation, miscommunication, and misunderstanding; immersion in the virtual world; social presence and initiative and flow. Overall, the study showed that participants experienced positive emotional and social outcomes. We argue that this was achieved as a consequence of EVA Park being not only accessible but also a varied and entertaining environment within which PWA experienced both the realistic and the quirky while engaging with others and having fun.

Qualitative study of affective identities in dementia patients for the design of cognitive assistive technologies

Our overall aim is to develop an emotionally intelligent cognitive assistant (ICA) to help older adults with Alzheimer's disease (AD) to complete activities of daily living more independently. For improved adoption, such a system should take into account how individuals feel about who they are. This paper investigates different affective identities found in older care home residents with AD, leading to a computational characterization of these aspects and, thus, tailored prompts to each specific individual's identity in a way that potentially ensures smoother and more effective uptake and response. We report on a set of qualitative interviews with 12 older adult care home residents and caregivers. The interview covered life domains (family, origin, occupation, etc.), and feelings related to the ICA. All interviews were transcribed and analyzed to extract a set of affective identities, coded according to the social–psychological principles of affect control theory (ACT). Preliminary results show that a set of identities can be extracted for each participant (e.g. father, husband). Furthermore, our results provide support for the proposition that, while identities grounded in memories fade as a person loses their memory, habitual aspects of identity that reflect the overall “persona” may persist longer, even without situational context.

Evaluating the Benefits of Aphasia Intervention Delivered in Virtual Reality: Results of a Quasi-Randomised Study

INTRODUCTION

This study evaluated an intervention for people with aphasia delivered in a novel virtual reality platform called EVA Park. EVA Park contains a number of functional and fantastic locations and allows for interactive communication between multiple users. Twenty people with aphasia had 5 weeks' intervention, during which they received daily language stimulation sessions in EVA Park from a support worker. The study employed a quasi randomised design, which compared a group that received immediate intervention with a waitlist control group. Outcome measures explored the effects of intervention on communication and language skills, communicative confidence and feelings of social isolation. Compliance with the intervention was also explored through attrition and usage data.

RESULTS

There was excellent compliance with the intervention, with no participants lost to follow up and most (18/20) receiving at least 88% of the intended treatment dose. Intervention brought about significant gains on a measure of functional communication. Gains were achieved by both groups of participants, once intervention was received, and were well maintained. Changes on the measures of communicative confidence and feelings of social isolation were not achieved. Results are discussed with reference to previous aphasia therapy findings.

Computational Analysis and Simulation of Empathic Behaviors: a Survey of Empathy Modeling with Behavioral Signal Processing Framework

Empathy is an important psychological process that facilitates human communication and interaction. Enhancement of empathy has profound significance in a range of applications. In this paper, we review emerging directions of research on computational analysis of empathy expression and perception as well as empathic interactions, including their simulation. We summarize the work on empathic expression analysis by the targeted signal modalities (e.g., text, audio, and facial expressions). We categorize empathy simulation studies into theory-based emotion space modeling or application-driven user and context modeling. We summarize challenges in computational study of empathy including conceptual framing and understanding of empathy, data availability, appropriate use and validation of machine learning techniques, and behavior signal processing. Finally, we propose a unified view of empathy computation and offer a series of open problems for future research.

Script Templates: A Practical Approach to Script Training in Aphasia

PURPOSE

Script training for aphasia involves repeated practice of relevant phrases and sentences that, when mastered, can potentially be used in other communicative situations. Although an increasingly popular approach, script development can be time-consuming. We provide a detailed summary of the evidence supporting this approach. We then describe a method in which scripts at various levels of difficulty are created by systematically manipulating readability and grammatical and semantic components. We assess the appropriateness of using these template-based scripts with persons with aphasia of differing severities.

METHOD

We evaluated the oral reading performance of eight individuals with chronic non-fluent aphasia on scripts developed from the templates. Scripts were either of high or low difficulty relative to their aphasia severity, and personalized by inserting the participant's town and the name of an acquaintance. Oral reading probes were taken on three separate days within a week and performance within and across participants was examined.

RESULTS

Regardless of the participant's aphasia severity, scripts in the low-difficulty condition were read with significantly greater accuracy than scripts in the high-difficulty condition.

DISCUSSION

These findings support the use of graded script templates to ensure that appropriately challenging scripts are delivered to persons with aphasia for both clinical practice and research.

Impact of Personal Relevance on Acquisition and Generalization of Script Training for Aphasia: A Preliminary Analysis

PURPOSE

The importance of personalization in script training in aphasia has been assumed but never tested. This study compared acquisition and generalization of personally relevant versus generic words or phrases appearing in the same scripts.

METHOD

Eight individuals (6 men; 2 women) with chronic aphasia received 3 weeks of intensive computer-based script training. For each participant, 2 scripts, a trained and an untrained generalization script, were embedded with 4 personally relevant word choices and 2-4 generic items that were similar across participants. Scripts were probed for accuracy at baseline and posttreatment. Significance testing was conducted on baseline and posttreatment scores, and for gains in personally relevant versus generic items. Effect sizes were computed.

RESULTS

Both personally relevant and generic items improved significantly on trained scripts. Improvements on untrained scripts were smaller, with only personally relevant items reaching significance. There was no significant difference on gains made on personalized versus generic items for trained scripts (p = .059), but the effect size was large (d = 0.90). For generalization scripts, this effect was small (d = 0.25) and nonsignificant.

CONCLUSIONS

Personally relevant words and phrases were acquired, although not generalized, more successfully than generic words and phrases. Data supports the importance of personalization in script training, but the degree of that importance requires further investigation.