Socially Assistive Robot for Stroke Rehabilitation: A Long-Term in-the-Wild Pilot Randomized Controlled Trial

Socially assistive robots (SARs) have been suggested as a platform for post-stroke training. It is not yet known whether long-term interaction with a SAR can lead to an improvement in the functional ability of individuals post-stroke. The aim of this pilot study was to compare the changes in motor ability and quality of life following a long-term intervention for upper-limb rehabilitation of post-stroke individuals using three approaches: 1) training with a SAR in addition to usual care; 2) training with a computer in addition to usual care; and 3) usual care with no additional intervention. Thirty-three post-stroke patients with moderate-severe to mild impairment were randomly allocated into three groups: two intervention groups - one with a SAR (ROBOT group) and one with a computer (COMPUTER group) - and one control group with no intervention (CONTROL group). The intervention sessions took place three times/week, for a total of 15 sessions/participant; The study was conducted over a period of two years, during which 306 sessions were held. Twenty-six participants completed the study. Participants in the ROBOT group significantly improved in their kinematic and clinical measures which included smoothness of movement, action research arm test (ARAT), and Fugl-Meyer upper-extremity assessment (FMA-UE). No significant improvement in these measures was found in the COMPUTER or the control groups. 100% of the participants in the SAR group gained improvement which reached - or exceeded - the minimal clinically important difference in the ARAT, the gold standard for upper-extremity activity performance post-stroke. This study demonstrates both the feasibility and the clinical benefit of using a SAR for long-term interaction with post-stroke individuals as part of their rehabilitation program. Trial Registration: ClinicalTrials.gov NCT03651063.

Barriers and Facilitators of Vestibular Rehabilitation: Patients and Physiotherapists' Perspectives

BACKGROUND AND PURPOSE

Poor adherence to vestibular rehabilitation protocols is a known barrier to optimal care. Vestibular clinicians' comprehensive understanding of the barriers and facilitators to vestibular home exercise programs (VHEP) is a key element to achieving optimal care in the context of vestibular rehabilitation. The aims of this study are as follows: (1) to identify primary barriers and facilitators to VHEP from the perspective of patients with vestibular dysfunction and vestibular physical therapists (PTs); and (2) to provide strategies for clinicians to improve adherence and outcomes of VHEP.

METHODS

A qualitative research with single-session focus groups conducted separately for: (1) patients with vestibular disorders and (2) vestibular PTs. Six focus groups were conducted, 3 for each population, with a total of 39 participants. An online survey was conducted to evaluate the estimates of adherence rates, followed by a structured discussion over barriers and facilitators to VHEP as perceived by patients and PTs. Thematic data analyses were performed using a mixed deductive-inductive approach.

RESULTS

Eighteen patients with vestibular disorders and 21 experienced vestibular PTs participated in this study. Six barrier categories and 5 facilitator categories were identified. Barriers included motivation aspects, provocation of symptoms, time management, associated impairments, missing guidance and feedback, and psychosocial factors. Facilitators included motivation aspects, time management, patient education and exercise instructions, exercise setting, and associated symptom management.

DISCUSSION AND CONCLUSIONS

Clinicians who prescribe home exercise to patients with vestibular disorders can use this information about common barriers and facilitators for patient education and to provide optimal care and improve rehabilitation outcomes.

VIDEO ABSTRACT AVAILABLE

for more insights from the authors (see the video, Supplemental Digital Content, available at: http://links.lww.com/JNPT/A467).

Corrigendum: Correlation between kinetic and kinematic measures, clinical tests and subjective self-evaluation questionnaires of the affected upper limb in people after stroke

[This corrects the article DOI: 10.3389/fnins.2023.1264513.].

Correlation between kinetic and kinematic measures, clinical tests and subjective self-evaluation questionnaires of the affected upper limb in people after stroke

Introduction

Assessment of stroke recovery should include multiple sources of information in order to obtain a complete understanding of the individual's rehabilitation progress. Self-evaluation questionnaires' scores do not always correspond to the scores of commonly used clinical evaluation tools. The purpose of this study was to assess the relationship between self-evaluation questionnaires, clinical tests, and kinematic and kinetic analyses of the affected upper limb after stroke, and to determine the correlation between these measures and self-reported general function 2-4 years after the stroke.

Methods

Twenty-six subjects recovering from stroke were included in the study. Spearman's correlation coefficient was used to measure the correlation between Stroke Impact Scale (SIS), Motor activity Log (MAL), Fugl-Meyer Assessment (FMA) and Action Reach Arm Test (ARAT) scores, and kinematic and kinetic analyses. A logistic regression was used to assess the extent to which these measures may predict the participants' functional self-reported status 2-4 years post stroke.

Results

Sections regarding hand function, hand force and general ADL of the self-evaluation questionnaires correlated with kinematic variables. However, only questionnaires that focus on hand function correlated with clinical tests. Mean and maximal hand velocity had the strongest correlations with self-evaluation questionnaires and with the clinical tests, more than other kinematic variables. Self-evaluation questionnaires and clinical tests were found to be correlated with hand kinetic metrics force-to-time ratio and number of force peaks. SIS hand force domain, mean velocity and maximal velocity predicted self-reported general function 2-4 years after the stroke.

Conclusion

Self-evaluation questionnaires should be considered for wider use in the clinical evaluation of a patient's stroke recovery, since they add important information on the individual's functional status, which is not reflected in the clinical tests.

"Robots for good": Ten defining questions

Ten questions to guide reflection and assessment of the "good" in robotics projects are suggested.

A novel socially assistive robotic platform for cognitive-motor exercises for individuals with Parkinson's Disease: a participatory-design study from conception to feasibility testing with end users

The potential of socially assistive robots (SAR) to assist in rehabilitation has been demonstrated in contexts such as stroke and cardiac rehabilitation. Our objective was to design and test a platform that addresses specific cognitive-motor training needs of individuals with Parkinson's disease (IwPD). We used the participatory design approach, and collected input from a total of 62 stakeholders (IwPD, their family members and clinicians) in interviews, brainstorming sessions and in-lab feasibility testing of the resulting prototypes. The platform we developed includes two custom-made mobile desktop robots, which engage users in concurrent cognitive and motor tasks. IwPD (n = 16) reported high levels of enjoyment when using the platform (median = 5/5) and willingness to use the platform in the long term (median = 4.5/5). We report the specifics of the hardware and software design as well as the detailed input from the stakeholders.

Ethical Considerations in Child-Robot Interactions

Social robots hold promise in augmenting education, rehabilitative care, and leisure activities for children. Despite findings suggesting various benefits of social robot use in schools, clinics, and homes, stakeholders have voiced concerns about the potential social and emotional effects of children engaging in long-term interactions with robots. Given the challenges of conducting large long-term studies of child-robot interaction (CRI), little is known about the impact of CRI on children's socio-emotional development. Here we summarize the literature on predictions and expectations of teachers, parents, therapists, and children regarding the effects of CRI on children's socio-emotional functioning and skill building. We then highlight the limited body of empirical research examining how CRI affects children's social behavior and emotional expression, and we provide a summary of available questionnaires for measuring socio-emotional constructs relevant to CRI. We conclude with design recommendations for research studies aimed at better understanding the effects of CRI, before social robots become more ubiquitous. This review is relevant to researchers, educators, roboticists, and clinicians interested in designing and using social robots with developmental populations.

A Socially Assistive Robot for Stroke Patients: Acceptance, Needs, and Concerns of Patients and Informal Caregivers

Stroke patients often contend with long-term physical challenges that require treatment and support from both formal and informal caregivers. Socially Assistive Robots (SARs) can assist patients in their physical rehabilitation process and relieve some of the burden on the informal caregivers, such as spouses and family members. We collected and analyzed information from 23 participants (11 stroke patients and 12 informal caregivers) who participated in a total of six focus-group discussions. The participants responded to questions regarding using a SAR to promote physical exercises during the rehabilitation process: (a) the advantages and disadvantages of doing so; (b) specific needs that they wish a SAR would address; (c) patient-specific adaptations they would propose to include; and (d) concerns they had regarding the use of such technology in stroke rehabilitation. We found that the majority of the participants in both groups were interested in experiencing the use of a SAR for rehabilitation, in the clinic and at home. Both groups noted the advantage of having the constant presence of a motivating entity with whom they can practice their rehabilitative exercises. The patients noted how such a device can assist formal caregivers in managing their workload, while the informal caregivers indicated that such a system could ease their own workload and sense of burden. The main disadvantages that participants noted related to the robot not possessing human abilities, such as the ability to hold a conversation, to physically guide the patient's movements, and to express or understand emotions. We anticipate that the data collected in this study-input from the patients and their family members, including the similarities and differences between their points of view-will aid in improving the development of SARs for rehabilitation, so that they can better suit people who have had a stroke, and meet their individual needs.

Extended Interviews with Stroke Patients Over a Long-Term Rehabilitation Using Human–Robot or Human–Computer Interactions

Socially assistive robots (SARs) have been proposed to assist post-stroke patients in performing their exercise during their rehabilitation process, with the trust in the robot identified as an important factor in human–robot interaction. In the current study, we aimed to identify and characterize factors that influence post-stroke patients’ trust in a robot-operated and a computer-operated rehabilitation platform during and after a long-term experience with the platform. We conducted 29 interviews with 16 stroke patients who underwent a long-term rehabilitation process, assisted by either a SAR or a computer interface. The intervention lasted 5–7 weeks per patient, for a total of 229 sessions over 18 months. By using a qualitative research method—extended interviews “in the wild” with stroke patients, over a long-term rehabilitation process—our study reveals users’ perspectives regarding factors affecting trust in the SAR or in the computer interface during their rehabilitation process. The results support the assertion that SARs have an added value in the rehabilitative care of stroke patients; It appears that personal characteristics, such as age and gender, have an effect on the users’ acceptance of a non-human operator as a practice assistant. Our findings support the notion that SARs augment rehabilitative therapies beyond a standard computer; Importantly, patients appreciated different aspects of the non-human operator in the two groups: In the SAR group, users preferred its functional performance over its anthropomorphized social skills; In the Computer group, users highlighted its contribution to the training of their memory skills.

Automating provision of feedback to stroke patients with and without information on compensatory movements: A pilot study

Providing effective feedback to patients in a rehabilitation training program is essential. As technologies are being developed to support patient training, they need to be able to provide the users with feedback on their performance. As there are various aspects on which feedback can be given (e.g., task success and presence of compensatory movements), it is important to ensure that users are not overwhelmed by too much information given too frequently by the assistive technology. We created a rule-based set of guidelines for the desired hierarchy, timing, and content of feedback to be used when stroke patients train with an upper-limb exercise platform which we developed. The feedback applies to both success on task completion and to the execution of compensatory movements, and is based on input collected from clinicians in a previous study. We recruited 11 stroke patients 1–72 months from injury onset. Ten participants completed the training; each trained with the rehabilitation platform in two configurations: with motor feedback (MF) and with no motor feedback (control condition) (CT). The two conditions were identical, except for the feedback content provided: in both conditions they received feedback on task success; in the MF condition they also received feedback on making undesired compensatory movements during the task. Participants preferred the configuration that provided feedback on both task success and quality of movement (MF). This pilot experiment demonstrates the feasibility of a system providing both task-success and movement-quality feedback to patients based on a decision tree which we developed.

Interaction Matters: The Effect of Touching the Social Robot PARO on Pain and Stress is Stronger When Turned ON vs. OFF

Social touch between humans, as well as between humans and animals, was previously found to reduce pain and stress. We previously reported that touching a social robot can also induce a reduction in pain ratings. However, it is unclear if the effect that touching a robot has on pain perception is due to its appearance and its pleasant touch, or due to its ability to socially interact with humans. In the current experiment, we aimed to assess the contribution of the interactive quality to pain perception. We assessed the effect of touching the social robot PARO on mild and strong pain ratings and on stress perception, on a total of 60 healthy young participants. The robot either interacted with participants (ON group, n = 30) or was turned off (OFF group, n = 30). Touching the robot induced a decrease in mild pain ratings (compared to baseline) only in the ON group while strong pain ratings decreased similarly in both the ON and the OFF groups. The decrease in mild pain ratings in the ON group was significantly greater in participants with a higher positive perception of the interaction with PARO. We conclude that part of the effect that touching the robot has on pain stems from its interactive features.

Emerging Roles for Social Robots in Rehabilitation

Insights from social and cognitive neuroscience should inform the design of socially assistive robots for neurorehabilitation as novel roles emerge for them in human-human interactions.

A robot goes to rehab: a novel gamified system for long-term stroke rehabilitation using a socially assistive robot-methodology and usability testing

Background

Socially assistive robots (SARs) have been proposed as a tool to help individuals who have had a stroke to perform their exercise during their rehabilitation process. Yet, to date, there are no data on the motivating benefit of SARs in a long-term interaction with post-stroke patients.

Methods

Here, we describe a robot-based gamified exercise platform, which we developed for long-term post-stroke rehabilitation. The platform uses the humanoid robot Pepper, and also has a computer-based configuration (with no robot). It includes seven gamified sets of exercises, which are based on functional tasks from the everyday life of the patients. The platform gives the patients instructions, as well as feedback on their performance, and can track their performance over time. We performed a long-term patient-usability study, where 24 post-stroke patients were randomly allocated to exercise with this platform—either with the robot or the computer configuration—over a 5–7 week period, 3 times per week, for a total of 306 sessions.

Results

The participants in both groups reported that this rehabilitation platform addressed their arm rehabilitation needs, and they expressed their desire to continue training with it even after the study ended. We found a trend for higher acceptance of the system by the participants in the robot group on all parameters; however, this difference was not significant. We found that system failures did not affect the long-term trust that users felt towards the system.

Conclusions

We demonstrated the usability of using this platform for a long-term rehabilitation with post-stroke patients in a clinical setting. We found high levels of acceptance of both platform configurations by patients following this interaction, with higher ratings given to the SAR configuration. We show that it is not the mere use of technology that increases the motivation of the person to practice, but rather it is the appreciation of the technology’s effectiveness and its perceived contribution to the rehabilitation process. In addition, we provide a list of guidelines that can be used when designing and implementing other technological tools for rehabilitation.

Trial registration: This trial is registered in the NIH ClinicalTrials.gov database. Registration number NCT03651063, registration date 21.08.2018. https://clinicaltrials.gov/ct2/show/NCT03651063.

Touching the social robot PARO reduces pain perception and salivary oxytocin levels

Human-human social touch improves mood and alleviates pain. No studies have so far tested the effect of human-robot emotional touch on experimentally induced pain ratings, on mood and on oxytocin levels in healthy young adults. Here, we assessed the effect of touching the robot PARO on pain perception, on mood and on salivary oxytocin levels, in 83 young adults. We measured their perceived pain, happiness state, and salivary oxytocin. For the 63 participants in the PARO group, pain was assessed in three conditions: Baseline, Touch (touching PARO) and No-Touch (PARO present). The control group (20 participants) underwent the same measurements without ever encountering PARO. There was a decrease in pain ratings and in oxytocin levels and an increase in happiness ratings compared to baseline only in the PARO group. The Touch condition yielded a larger decrease in pain ratings compared to No-Touch. These effects correlated with the participants' positive perceptions of the interaction with PARO. Participants with higher perceived ability to communicate with PARO experienced a greater hypoalgesic effect when touching PARO. We show that human-robot social touch is effective in reducing pain ratings, improving mood and - surprisingly - reducing salivary oxytocin levels in adults.

Implicit task switching in Parkinson's disease is preserved when on medication

People with Parkinson's disease have been shown to have difficulty switching between movement plans. In the great majority of studies, the need to switch between tasks was made explicitly. Here, we tested whether people with Parkinson's disease, taking their normal medication, have difficulty switching between implicitly specified tasks. We further examined whether this switch is performed predictively or reactively. Twenty five people with Parkinson's disease continuously increased or decreased the frequency of their arm movements, inducing an abrupt-but unaware-switch between rhythmic movements (at high frequencies) and discrete movements (at low frequencies). We tested whether that precipitous change was performed reactively or predictively. We found that 56% of participants predictively switched between the two movement types. The ability of people with Parkinson's disease, taking their regular medication, to predictively control their movements on implicit tasks is thus preserved.

Trust in socially assistive robots: Considerations for use in rehabilitation

Incorporation of social robots into rehabilitation calls for understanding what factors affect user motivation and success of the interaction. Trust between the user and the robot has been identified as important in human-robot interaction and in human-human interactions in therapy. Trust has been studied in the context of automation technology, (e.g., autonomous cars), but not in the context of social robots for rehabilitation. In this narrative review, we address the unique patient-clinician-robot triad, and argue that this context calls for specific design features in order to foster trust with the users. We review pertinent methods for measuring trust, and studies demonstrating that culture, prior experience and propensity-to-trust affect to what extent users trust robots. We suggest design guidelines for fostering trust and methods for measuring trust in human-robot interactions in rehabilitation. We stress the need to create measures of trust that are accessible to people who suffer from speech or cognitive impairments. This review is pertinent to researchers, roboticists, and clinicians interested in designing and using social robots for rehabilitation.

Sooner Rather Than Later: Precrastination Rather Than Procrastination

Putting things off as long as possible (procrastination) is a well-known tendency. Less well known is the tendency to attempt to get things done as soon as possible, even if that involves extra effort ( precrastination). Since its discovery in 2014, precrastination has been demonstrated in humans and animals and has recently been revealed in an analogous tendency called the mere-urgency effect. Trying to get things done as soon as one can may reflect optimal foraging, but another less obvious factor may also contribute—reducing cognitive demands associated with having to remember what to do when. Individual differences may also play a role. Understanding precrastination will have important implications for explaining why hurrying happens as often as it does and may help reduce the chance that haste makes waste.

Smooth leader or sharp follower? Playing the mirror game with a robot

Background:

The increasing number of opportunities for human-robot interactions in various settings, from industry through home use to rehabilitation, creates a need to understand how to best personalize human-robot interactions to fit both the user and the task at hand. In the current experiment, we explored a human-robot collaborative task of joint movement, in the context of an interactive game.

Objective:

We set out to test people’s preferences when interacting with a robotic arm, playing a leader-follower imitation game (the mirror game).

Methods:

Twenty two young participants played the mirror game with the robotic arm, where one player (person or robot) followed the movements of the other. Each partner (person and robot) was leading part of the time, and following part of the time. When the robotic arm was leading the joint movement, it performed movements that were either sharp or smooth, which participants were later asked to rate.

Results:

The greatest preference was given to smooth movements. Half of the participants preferred to lead, and half preferred to follow. Importantly, we found that the movements of the robotic arm primed the subsequent movements performed by the participants.

Conclusion:

The priming effect by the robot on the movements of the human should be considered when designing interactions with robots. Our results demonstrate individual differences in preferences regarding the role of the human and the joint motion path of the robot and the human when performing the mirror game collaborative task, and highlight the importance of personalized human-robot interactions.

Robotic gaming prototype for upper limb exercise: Effects of age and embodiment on user preferences and movement

Background:

Effective human-robot interactions in rehabilitation necessitates an understanding of how these should be tailored to the needs of the human. We report on a robotic system developed as a partner on a 3-D everyday task, using a gamified approach.

Objectives:

To: (1) design and test a prototype system, to be ultimately used for upper-limb rehabilitation; (2) evaluate how age affects the response to such a robotic system; and (3) identify whether the robot’s physical embodiment is an important aspect in motivating users to complete a set of repetitive tasks.

Methods:

62 healthy participants, young (<30 yo) and old (>60 yo), played a 3D tic-tac-toe game against an embodied (a robotic arm) and a non-embodied (a computer-controlled lighting system) partner. To win, participants had to place three cups in sequence on a physical 3D grid. Cup picking-and-placing was chosen as a functional task that is often practiced in post-stroke rehabilitation. Movement of the participants was recorded using a Kinect camera.

Results:

The timing of the participants’ movement was primed by the response time of the system: participants moved slower when playing with the slower embodied system (p = 0.006). The majority of participants preferred the robot over the computer-controlled system. Slower response time of the robot compared to the computer-controlled one only affected the young group’s motivation to continue playing.

Conclusion:

We demonstrated the feasibility of the system to encourage the performance of repetitive 3D functional movements, and track these movements. Young and old participants preferred to interact with the robot, compared with the non-embodied system. We contribute to the growing knowledge concerning personalized human-robot interactions by (1) demonstrating the priming of the human movement by the robotic movement – an important design feature, and (2) identifying response-speed as a design variable, the importance of which depends on the age of the user.

Social robots in rehabilitation: A question of trust

Social robots can help meet the growing need for rehabilitation assistance; measures for creating and maintaining trust in human-robot interactions should be priorities when designing social robots for rehabilitation.

Improvement in upper-limb UPDRS motor scores following fast-paced arm exercise: A pilot study

BACKGROUND

The symptoms of patients with Parkinson's disease (PD) have been shown to improve when they perform fast-paced rhythmic cycling movements with their lower limbs.

OBJECTIVE

Our goal in this pilot experiment was to test the feasibility and the benefits of a short exercise program involving fast-paced rhythmic movements of the upper limb for patients with PD.

METHODS

We used an experimental procedure that elicits large, fast-paced movements by the participants without the direct instructions to do so by the experimenter. Ten participants with PD (71.0±6.5 years old) performed a 50-min fast-paced rhythmic exercise of the upper limb after withdrawal from PD medication for at least 12 hours.

RESULTS

Participants improved their kinematic performance, in terms of accuracy and combined speed and amplitude (p < 0.02), as well as their upper-limb MDS-UPDRS motor scores (p = 0.023).

CONCLUSIONS

The results demonstrate the feasibility of using the described apparatus to perform an exercise session of approximately 50 min with both arms, and give a preliminary indication of the potential benefit of such an exercise program.

Changes in Predictive Task Switching with Age and with Cognitive Load

Predictive control of movement is more efficient than feedback-based control, and is an important skill in everyday life. We tested whether the ability to predictively control movements of the upper arm is affected by age and by cognitive load. A total of 63 participants were tested in two experiments. In both experiments participants were seated, and controlled a cursor on a computer screen by flexing and extending their dominant arm. In Experiment 1, 20 young adults and 20 older adults were asked to continuously change the frequency of their horizontal arm movements, with the goal of inducing an abrupt switch between discrete movements (at low frequencies) and rhythmic movements (at high frequencies). We tested whether that change was performed based on a feed-forward (predictive) or on a feedback (reactive) control. In Experiment 2, 23 young adults performed the same task, while being exposed to a cognitive load half of the time via a serial subtraction task. We found that both aging and cognitive load diminished, on average, the ability of participants to predictively control their movements. Five older adults and one young adult under a cognitive load were not able to perform the switch between rhythmic and discrete movement (or vice versa). In Experiment 1, 40% of the older participants were able to predictively control their movements, compared with 70% in the young group. In Experiment 2, 48% of the participants were able to predictively control their movements with a cognitively loading task, compared with 70% in the no-load condition. The ability to predictively change a motor plan in anticipation of upcoming changes may be an important component in performing everyday functions, such as safe driving and avoiding falls.

Motor errors lead to enhanced performance in older adults

Young individuals make larger and faster forearm movements when visual feedback about the movement is not available, compared to when it is. We set out to test whether this behavior persists with aging. We tested 40 participants, 20 in each age group - young and old, on a task that required making rhythmic movements of the forearm with and without visual feedback. Surprisingly, we found that older adults increased the speed and the amplitude of their movements to an even greater extent than did the young adults. Furthermore, we found that the increase in speed and amplitude during the non-vision trial segments improved their performance on the task, and they were able to leverage the change in these movement parameters (speed and amplitude) to improve their performance during subsequent trial segments that did include visual feedback. The improvement in accuracy on the task was accompanied by a decrease in path variability. The results indicate that older adults can adapt their movement parameters to enhance performance following a motor perturbation. They further suggest that motor variability in old age can be advantageous under certain circumstances.

Other ways of seeing: From behavior to neural mechanisms in the online "visual" control of action with sensory substitution

Vision is the dominant sense for perception-for-action in humans and other higher primates. Advances in sight restoration now utilize the other intact senses to provide information that is normally sensed visually through sensory substitution to replace missing visual information. Sensory substitution devices translate visual information from a sensor, such as a camera or ultrasound device, into a format that the auditory or tactile systems can detect and process, so the visually impaired can see through hearing or touch. Online control of action is essential for many daily tasks such as pointing, grasping and navigating, and adapting to a sensory substitution device successfully requires extensive learning. Here we review the research on sensory substitution for vision restoration in the context of providing the means of online control for action in the blind or blindfolded. It appears that the use of sensory substitution devices utilizes the neural visual system; this suggests the hypothesis that sensory substitution draws on the same underlying mechanisms as unimpaired visual control of action. Here we review the current state of the art for sensory substitution approaches to object recognition, localization, and navigation, and the potential these approaches have for revealing a metamodal behavioral and neural basis for the online control of action.

Aging and Sensory Substitution in a Virtual Navigation Task

Virtual environments are becoming ubiquitous, and used in a variety of contexts–from entertainment to training and rehabilitation. Recently, technology for making them more accessible to blind or visually impaired users has been developed, by using sound to represent visual information. The ability of older individuals to interpret these cues has not yet been studied. In this experiment, we studied the effects of age and sensory modality (visual or auditory) on navigation through a virtual maze. We added a layer of complexity by conducting the experiment in a rotating room, in order to test the effect of the spatial bias induced by the rotation on performance. Results from 29 participants showed that with the auditory cues, it took participants a longer time to complete the mazes, they took a longer path length through the maze, they paused more, and had more collisions with the walls, compared to navigation with the visual cues. The older group took a longer time to complete the mazes, they paused more, and had more collisions with the walls, compared to the younger group. There was no effect of room rotation on the performance, nor were there any significant interactions among age, feedback modality and room rotation. We conclude that there is a decline in performance with age, and that while navigation with auditory cues is possible even at an old age, it presents more challenges than visual navigation.

Integration and binding in rehabilitative sensory substitution: Increasing resolution using a new Zooming-in approach

PURPOSE

To visually perceive our surroundings we constantly move our eyes and focus on particular details, and then integrate them into a combined whole. Current visual rehabilitation methods, both invasive, like bionic-eyes and non-invasive, like Sensory Substitution Devices (SSDs), down-sample visual stimuli into low-resolution images. Zooming-in to sub-parts of the scene could potentially improve detail perception. Can congenitally blind individuals integrate a 'visual' scene when offered this information via different sensory modalities, such as audition? Can they integrate visual information -perceived in parts - into larger percepts despite never having had any visual experience?

METHODS

We explored these questions using a zooming-in functionality embedded in the EyeMusic visual-to-auditory SSD. Eight blind participants were tasked with identifying cartoon faces by integrating their individual components recognized via the EyeMusic's zooming mechanism.

RESULTS

After specialized training of just 6-10 hours, blind participants successfully and actively integrated facial features into cartooned identities in 79±18% of the trials in a highly significant manner, (chance level 10% ; rank-sum P <  1.55E-04).

CONCLUSIONS

These findings show that even users who lacked any previous visual experience whatsoever can indeed integrate this visual information with increased resolution. This potentially has important practical visual rehabilitation implications for both invasive and non-invasive methods.

The "EyeCane", a new electronic travel aid for the blind: Technology, behavior & swift learning

PURPOSE

Independent mobility is one of the most pressing problems facing people who are blind. We present the EyeCane, a new mobility aid aimed at increasing perception of environment beyond what is provided by the traditional White Cane for tasks such as distance estimation, navigation and obstacle detection.

METHODS

The "EyeCane" enhances the traditional White Cane by using tactile and auditory output to increase detectable distance and angles. It circumvents the technical pitfalls of other devices, such as weight, short battery life, complex interface schemes, and slow learning curve. It implements multiple beams to enables detection of obstacles at different heights, and narrow beams to provide active sensing that can potentially increase the user's spatial perception of the environment. Participants were tasked with using the EyeCane for several basic tasks with minimal training.

RESULTS

Blind and blindfolded-sighted participants were able to use the EyeCane successfully for distance estimation, simple navigation and simple obstacle detection after only several minutes of training.

CONCLUSIONS

These results demonstrate the EyeCane's potential for mobility rehabilitation. The short training time is especially important since available mobility training resources are limited, not always available, and can be quite expensive and/or entail long waiting periods.

The effect of extended sensory range via the EyeCane sensory substitution device on the characteristics of visionless virtual navigation

Mobility training programs for helping the blind navigate through unknown places with a White-Cane significantly improve their mobility. However, what is the effect of new assistive technologies, offering more information to the blind user, on the underlying premises of these programs such as navigation patterns? We developed the virtual-EyeCane, a minimalistic sensory substitution device translating single-point-distance into auditory cues identical to the EyeCane's in the real world. We compared performance in virtual environments when using the virtual-EyeCane, a virtual-White-Cane, no device and visual navigation. We show that the characteristics of virtual-EyeCane navigation differ from navigation with a virtual-White-Cane or no device, and that virtual-EyeCane users complete more levels successfully, taking shorter paths and with less collisions than these groups, and we demonstrate the relative similarity of virtual-EyeCane and visual navigation patterns. This suggests that additional distance information indeed changes navigation patterns from virtual-White-Cane use, and brings them closer to visual navigation.

EyeMusic: Introducing a "visual" colorful experience for the blind using auditory sensory substitution

PURPOSE

Sensory-substitution devices (SSDs) provide auditory or tactile representations of visual information. These devices often generate unpleasant sensations and mostly lack color information. We present here a novel SSD aimed at addressing these issues.

METHODS

We developed the EyeMusic, a novel visual-to-auditory SSD for the blind, providing both shape and color information. Our design uses musical notes on a pentatonic scale generated by natural instruments to convey the visual information in a pleasant manner. A short behavioral protocol was utilized to train the blind to extract shape and color information, and test their acquired abilities. Finally, we conducted a survey and a comparison task to assess the pleasantness of the generated auditory stimuli.

RESULTS

We show that basic shape and color information can be decoded from the generated auditory stimuli. High performance levels were achieved by all participants following as little as 2-3 hours of training. Furthermore, we show that users indeed found the stimuli pleasant and potentially tolerable for prolonged use.

CONCLUSIONS

The novel EyeMusic algorithm provides an intuitive and relatively pleasant way for the blind to extract shape and color information. We suggest that this might help facilitating visual rehabilitation because of the added functionality and enhanced pleasantness.

Color improves "visual" acuity via sound

Visual-to-auditory sensory substitution devices (SSDs) convey visual information via sound, with the primary goal of making visual information accessible to blind and visually impaired individuals. We developed the EyeMusic SSD, which transforms shape, location, and color information into musical notes. We tested the “visual” acuity of 23 individuals (13 blind and 10 blindfolded sighted) on the Snellen tumbling-E test, with the EyeMusic. Participants were asked to determine the orientation of the letter “E.” The test was repeated twice: in one test, the letter “E” was drawn with a single color (white), and in the other test, with two colors (red and white). In the latter case, the vertical line in the letter, when upright, was drawn in red, with the three horizontal lines drawn in white. We found no significant differences in performance between the blind and the sighted groups. We found a significant effect of the added color on the “visual” acuity. The highest acuity participants reached in the monochromatic test was 20/800, whereas with the added color, acuity doubled to 20/400. We conclude that color improves “visual” acuity via sound.

Increasing accessibility to the blind of virtual environments, using a virtual mobility aid based on the "EyeCane": feasibility study

Virtual worlds and environments are becoming an increasingly central part of our lives, yet they are still far from accessible to the blind. This is especially unfortunate as such environments hold great potential for them for uses such as social interaction, online education and especially for use with familiarizing the visually impaired user with a real environment virtually from the comfort and safety of his own home before visiting it in the real world. We have implemented a simple algorithm to improve this situation using single-point depth information, enabling the blind to use a virtual cane, modeled on the “EyeCane” electronic travel aid, within any virtual environment with minimal pre-processing. Use of the Virtual-EyeCane, enables this experience to potentially be later used in real world environments with identical stimuli to those from the virtual environment. We show the fast-learned practical use of this algorithm for navigation in simple environments.

Cross-sensory transfer of sensory-motor information: visuomotor learning affects performance on an audiomotor task, using sensory-substitution

Visual-to-auditory sensory-substitution devices allow users to perceive a visual image using sound. Using a motor-learning task, we found that new sensory-motor information was generalized across sensory modalities. We imposed a rotation when participants reached to visual targets, and found that not only seeing, but also hearing the location of targets via a sensory-substitution device resulted in biased movements. When the rotation was removed, aftereffects occurred whether the location of targets was seen or heard. Our findings demonstrate that sensory-motor learning was not sensory-modality-specific. We conclude that novel sensory-motor information can be transferred between sensory modalities.

Fast, accurate reaching movements with a visual-to-auditory sensory substitution device

PURPOSE

Visual sensory substitution devices (SSDs) use sound or touch to convey information that is normally perceived by vision. The primary focus of prior research using SSDs was the perceptual components of learning to use SSDs and their neural correlates. However, sensorimotor integration is critical in the effort to make SSDs relevant for everyday tasks, like grabbing a cup of coffee efficiently. The purpose of this study was to test the use of a novel visual-to-auditory SSD to guide a fast reaching movement.

METHODS

Using sound, the SSD device relays location, shape and color information. Participants were asked to make fast reaching movements to targets presented by the SSD.

RESULTS

After only a short practice session, blindfolded sighted participants performed fast and accurate movements to presented targets, which did not differ significantly from movements performed with visual feedback in terms of movement time, peak speed, and path length. A small but significant difference was found between the endpoint accuracy of movements under the two feedback conditions; remarkably, in both cases the average error was smaller than 0.5 cm.

CONCLUSIONS

Our findings combine with previous brain-imaging studies to support a theory of a modality-independent representation of spatial information. Task-specificity, rather than modality-specificity, of brain functions is crucially important for the rehabilitative use of SSDs in the blind and the visually impaired. We present the first direct comparison between movement trajectories performed with an SSD and ones performed under visual guidance. The accuracy level reached in this study demonstrates the potential applicability of using the visual-to-auditory SSD for performance of daily tasks which require fast, accurate reaching movements, and indicates a potential for rehabilitative use of the device.

The effects of rhythmicity and amplitude on transfer of motor learning

We perform rhythmic and discrete arm movements on a daily basis, yet the motor control literature is not conclusive regarding the mechanisms controlling these movements; does a single mechanism generate both movement types, or are they controlled by separate mechanisms? A recent study reported partial asymmetric transfer of learning from discrete movements to rhythmic movements. Other studies have shown transfer of learning between large-amplitude to small-amplitude movements. The goal of this study is to explore which aspect is important for learning to be transferred from one type of movement to another: rhythmicity, amplitude or both. We propose two hypotheses: (1) Rhythmic and discrete movements are generated by different mechanisms; therefore we expect to see a partial or no transfer of learning between the two types of movements; (2) Within each movement type (rhythmic/discrete), there will be asymmetric transition of learning from larger movements to smaller ones. We used a learning-transfer paradigm, in which 70 participants performed flexion/extension movements with their forearm, and switched between types of movement, which differed in amplitude and/or rhythmicity. We found partial transfer of learning between discrete and rhythmic movements, and an asymmetric transfer of learning from larger movements to smaller movements (within the same type of movement). Our findings suggest that there are two different mechanisms underlying the generation of rhythmic and discrete arm movements, and that practicing on larger movements helps perform smaller movements; the latter finding might have implications for rehabilitation.

Rhythmic movements are larger and faster but with the same frequency on removal of visual feedback

The brain controls rhythmic movement through neural circuits combining visual information with proprioceptive information from the limbs. Although rhythmic movements are fundamental to everyday activities the specific details of the responsible control mechanisms remain elusive. We tested 39 young adults who performed flexion/extension movements of the forearm. We provided them with explicit knowledge of the amplitude and the speed of their movements, whereas frequency information was only implicitly available. In a series of 3 experiments, we demonstrate a tighter control of frequency compared with amplitude or speed. We found that in the absence of visual feedback, movements had larger amplitude and higher peak speed while maintaining the same frequency as when visual feedback was available; this was the case even when participants were aware of performing overly large and fast movements. Finally, when participants were asked to modulate continuously movement frequency, but not amplitude, we found the local coefficient of variability of movement frequency to be lower than that of amplitude. We suggest that a misperception of the generated amplitude in the absence of visual feedback, coupled with a highly accurate perception of generated frequency, leads to the performance of larger and faster movements with the same frequency when visual feedback is not available. Relatively low local coefficient of variability of frequency in a task that calls for continuous change in movement frequency suggests that we tend to operate at a constant frequency at the expense of variation in amplitude and peak speed.

Rhythmic movement in Parkinson's disease: effects of visual feedback and medication state

Previous studies examining discrete movements of Parkinson's disease (PD) patients have found that in addition to performing movements that were slower than those of control participants, they exhibit specific deficits in movement coordination and in sensorimotor integration required to accurately guide movements. With medication, movement speed was normalized, but the coordinative aspects of movement were not. This led to the hypothesis that dopaminergic medication more readily compensates for intensive aspects of movement (such as speed), than for coordinative aspects (such as coordination of different limb segments) (Schettino et al., Exp Brain Res 168:186-202, 2006). We tested this hypothesis on rhythmic, continuous movements of the forearm. In our task, target peak speed and amplitude, availability of visual feedback, and medication state (on/off) were varied. We found, consistent with the discrete-movement results, that peak speed (intensive aspect) was normalized by medication, while accuracy, which required coordination of speed and amplitude modulation (coordinative aspect), was not normalized by dopaminergic treatment. However, our findings that amplitude, an intensive aspect of movement, was also not normalized by medication, suggests that a simple pathway gain increase does not act to remediate all intensive aspects of movement to the same extent. While it normalized movement peak speed, it did not normalize movement amplitude. Furthermore, we found that when visual feedback was not available, all participants (PD and controls) made faster movements. The effects of dopaminergic medication and availability of visual feedback on movement speed were additive. The finding that movement speed uniformly increased both in the PD and the control groups suggests that visual feedback may be necessary for calibration of peak speed, otherwise underestimated by the motor control system.

One dimensional Turing-like handshake test for motor intelligence

In the Turing test, a computer model is deemed to "think intelligently" if it can generate answers that are not distinguishable from those of a human. However, this test is limited to the linguistic aspects of machine intelligence. A salient function of the brain is the control of movement, and the movement of the human hand is a sophisticated demonstration of this function. Therefore, we propose a Turing-like handshake test, for machine motor intelligence. We administer the test through a telerobotic system in which the interrogator is engaged in a task of holding a robotic stylus and interacting with another party (human or artificial). Instead of asking the interrogator whether the other party is a person or a computer program, we employ a two-alternative forced choice method and ask which of two systems is more human-like. We extract a quantitative grade for each model according to its resemblance to the human handshake motion and name it "Model Human-Likeness Grade" (MHLG). We present three methods to estimate the MHLG. (i) By calculating the proportion of subjects' answers that the model is more human-like than the human; (ii) By comparing two weighted sums of human and model handshakes we fit a psychometric curve and extract the point of subjective equality (PSE); (iii) By comparing a given model with a weighted sum of human and random signal, we fit a psychometric curve to the answers of the interrogator and extract the PSE for the weight of the human in the weighted sum. Altogether, we provide a protocol to test computational models of the human handshake. We believe that building a model is a necessary step in understanding any phenomenon and, in this case, in understanding the neural mechanisms responsible for the generation of the human handshake.

Early switching between movement types: indication of predictive control?

In everyday life, we frequently alternate between performing discrete and rhythmic movements. When performing a periodic movement, two distinct movement types can be distinguished: highly harmonic vs. discrete-like. The harmonicity of the movement is used to classify it as one or the other. We asked: (1) whether the frequency at which a periodic movement is performed affects the harmonicity of the resultant movement; and (2) what underlies switching between these movement types. To answer these questions, we studied horizontal flexion/extension forearm movements in 13 young adults over a wide range of frequencies. Movements were performed either at a fixed frequency, or at gradually increasing or decreasing target frequencies. We found movement harmonicity to depend on the frequency of the movement. Furthermore, we found a reverse hysteresis behavior, where participants switched movement type in anticipation of the future-required frequency. These findings suggest that predictive control is employed in switching between movement types.

Non-monotonicity on a spatio-temporally defined cyclic task: evidence of two movement types?

We tested 23 healthy participants who performed rhythmic horizontal movements of the elbow. The required amplitude and frequency ranges of the movements were specified to the participants using a closed shape on a phase-plane display, showing angular velocity versus angular position, such that participants had to continuously control both the speed and the displacement of their forearm. We found that the combined accuracy in velocity and position throughout the movement was not a monotonic function of movement speed. Our findings suggest that specific combinations of required movement frequency and amplitude give rise to two distinct types of movements: one of a more rhythmic nature, and the other of a more discrete nature.