Making neurorehabilitation fun: Multiplayer training via damping forces balancing differences in skill levels

Automated Patient-Robot Task Assignment in a Simulated Stochastic Rehabilitation Gym

Rehabilitation after neurological injury can be provided by robots that help patients perform different exercises. Multiple such robots can be combined in a rehabilitation robot gym to allow multiple patients to perform a diverse range of exercises simultaneously. In pursuit of better multipatient supervision, we aim to develop an automated assignment system that assigns patients to different robots during a training session to maximize their skill development. Our previous work was designed for simplified simulated environments where each patient's skill development is known beforehand. The current work improves upon that work by changing the deterministic environment into a stochastic environment where part of the skill development is random and the assignment system must estimate each patient's predicted skill development using a neural network based on the patient's previous training success rate with that robot. These skill development estimates are used to create patient-robot assignments on a timestep-by-timestep basis to maximize the skill development of the patient group. Results from simplified simulation trials show that the schedules produced by our assignment system outperform multiple baseline schedules (e.g., schedules where patients never switch robots and schedules where patients only switch robots once halfway through the session). Additionally, we discuss how some of our simplifications could be addressed in the future.

Patients' physiological reactions to competitive rehabilitation therapies assisted by robotic devices

BACKGROUND

The aging of the population and the progressive increase in life expectancy in developed countries is leading to a high incidence of cerebrovascular diseases. Several studies have demonstrated that robot-assisted rehabilitation therapies combined with serious games can improve rehabilitation outcomes. Social interaction in the form of multiplayer games has been highlighted as a potential element to increase patient's motivation and exercise intensity, which professionals have described as one of the determining factors in maximizing rehabilitation outcomes. Despite this, it has not been widely studied. Physiological measures have been proven as an objective tool to evaluate patients' experience in robot-assisted rehabilitation environments. However, they have not been used to evaluate patients' experience in multiplayer robot-assisted rehabilitation therapies. The main objective of this study is to analyze whether the interpersonal interaction inherent in a competitive game mode affects the patients' physiological responses in robot-assisted rehabilitation environments.

METHODS

A total of 14 patients participated in this study. The results of a competitive game mode were compared with a single-player game mode with different difficulty levels. Exercise intensity and performance were measured through parameters extracted from the game and the information provided by the robotic rehabilitation platforms. The physiological response of patients in each game mode was measured by the heart rate (HR) and the galvanic skin response (GSR). Patients were asked to fill out the IMI and the overall experience questionnaire.

RESULTS

The exercise intensity results show that high-difficulty single-player game mode is similar in terms of intensity level to a competitive game mode, based on velocity values, reaction time and questionnaire results. However, the results of the physiological responses of the patients measured by GSR and HR are lower in the case of the competitive mode compared to the high-difficulty single-player game mode, obtaining results similar to those obtained in the low-difficulty single-player game mode.

CONCLUSIONS

Patients find the competitive game mode the most fun, which is also the mode they report experiencing the most effort and stress level. However, this subjective evaluation is not in line with the results of physiological responses. This study concludes that interpersonal interaction inherent to a competitive game mode influences patients' physiological responses. This could mean that social interaction is an important factor to consider when interpreting the results obtained from physiological measurements.

Automated patient-robot assignment for a robotic rehabilitation gym: a simplified simulation model

Background

A robotic rehabilitation gym can be defined as multiple patients training with multiple robots or passive sensorized devices in a group setting. Recent work with such gyms has shown positive rehabilitation outcomes; furthermore, such gyms allow a single therapist to supervise more than one patient, increasing cost-effectiveness. To allow more effective multipatient supervision in future robotic rehabilitation gyms, we propose an automated system that could dynamically assign patients to different robots within a session in order to optimize rehabilitation outcome.

Methods

As a first step toward implementing a practical patient-robot assignment system, we present a simplified mathematical model of a robotic rehabilitation gym. Mixed-integer nonlinear programming algorithms are used to find effective assignment and training solutions for multiple evaluation scenarios involving different numbers of patients and robots (5 patients and 5 robots, 6 patients and 5 robots, 5 patients and 7 robots), different training durations (7 or 12 time steps) and different complexity levels (whether different patients have different skill acquisition curves, whether robots have exit times associated with them). In all cases, the goal is to maximize total skill gain across all patients and skills within a session.

Results

Analyses of variance across different scenarios show that disjunctive and time-indexed optimization models significantly outperform two baseline schedules: staying on one robot throughout a session and switching robots halfway through a session. The disjunctive model results in higher skill gain than the time-indexed model in the given scenarios, and the optimization duration increases as the number of patients, robots and time steps increases. Additionally, we discuss how different model simplifications (e.g., perfectly known and predictable patient skill level) could be addressed in the future and how such software may eventually be used in practice.

Conclusions

Though it involves unrealistically simple scenarios, our study shows that intelligently moving patients between different rehabilitation robots can improve overall skill acquisition in a multi-patient multi-robot environment. While robotic rehabilitation gyms are not yet commonplace in clinical practice, prototypes of them already exist, and our study presents a way to use intelligent decision support to potentially enable more efficient delivery of technologically aided rehabilitation.

Artificial Intelligence-Driven Serious Games in Healthcare: A Scoping Review (Preprint).. [DOI: 10.2196/preprints.39840] [Citation(s) in RCA: 0] [Impact Index Per Article: 0]

BACKGROUND

Artificial Intelligence (AI)-driven serious games have been used in healthcare to offer a customizable and immersive experience. Summarizing the features of the current AI-driven serious games is very important to explore how they have been developed and used and their current state in order to plan on how to leverage them in the current and future healthcare needs.

OBJECTIVE

The current study aimed to explore the features of AI-driven serious games in healthcare as reported by previous research.

METHODS

We carried out a scoping review to achieve the above-mentioned objective. The most popular databases in information technology and health fields (e.g., MEDLINE and IEEE Xplore) were searched using keywords related to serious games and AI. These terms were selected based on the target intervention (i.e., AI) and the target disease (i.e., COVID-19). Two reviewers independently performed the study selection process. Three reviewers independently used Microsoft Excel to extract data from the included studies. A narrative approach was used for data synthesis.

RESULTS

The search process returned 1470 records. Of these records, 46 met all eligibility criteria. 60 different serious games were found in the included studies. Motor impairment was the most common health condition targeted by these serious games. Serious games in most of the studies were used for rehabilitation. The serious games in the majority of the included studies can be played by only single player. Most serious games were played on standalone devices (offline games). The most common genres of serious games were role-playing games, puzzle games, and platformer games. Unity was the most prominent game engine used to develop serious games. Personal computers (PCs) were the most common platforms used to play serious games. The most common algorithms used in the included studies were Support Vector Machine (SVM), Convolutional Neural Network (CNN), Artificial Neural Networks (ANN), and Random Forest (RF). The most common purposes of AI were the detection of disease and the evaluation of user's performance. The dataset size ranged from 36 to 795,600, with an average of about 52,124. The most common validation techniques used in the included studies were K-fold cross-validation and training test split validation. Accuracy was the most commonly used metric to evaluate the performance of AI models.

CONCLUSIONS

The last decade witnessed an increase in the development of AI-driven serious games for healthcare purposes and targeting various health conditions and leveraging multiple AI algorithms; this rising trend is expected to continue for years to come. While the evidence uncovered in this study shows promising applications of AI-driven serious games, larger and more rigorous, diverse, and robust studies may be needed to examine the efficacy and effectiveness of AI-driven serious games in different populations with different health conditions.

The effect of skill level matching in dyadic interaction on learning of a tracing task

Dyadic interaction between humans has gained great research interest in the last years. The effects of factors that influence the interaction, as e.g. roles or skill level matching, are still not well understood. In this paper, we further investigated the effect of skill level matching between partners on learning of a visuo-motor task. Understanding the effect of skill level matching is crucial for applications in collaborative rehabilitation. Fifteen healthy participants were asked to trace a path while being subjected to a visuo-motor rotation (Novice). The Novices were paired with a partner, forming one of the three Dyad Types: a) haptic connection to another Novice, b) haptic connection to an Expert (no visuo-motor rotation), or c) no haptic. The intervention consisted of a Familiarization phase, followed by a Training phase, in which the Novices were learning the task in the respective Dyad Type, and a Test phase in which the learning was assessed (haptic connection removed, if any). Results suggest that learning of the task with a haptic connection to an Expert was least beneficial. However, during the Training phase the dyads comprising an Expert clearly outperformed the dyads with matched skill levels. The results point towards the same direction as previous findings in literature and can be explained by current motor-learning theories. Future work needs to corroborate these preliminary results.

Effects of Different Opponent Types on Motivation and Exercise Intensity in a Competitive Arm Exercise Game

Objective: Competitive exercise games are popular in areas like rehabilitation and weight loss due to their positive effects on motivation. However, it is unclear whether a human opponent is necessary, as the same benefits may be achievable with a "human-like" computer-controlled opponent or a human who talks to the player without playing the game. Our objective was to compare four opponent types in a competitive exercise game: a simple computer opponent, "human-like" computer opponent, human opponent, and a simple computer opponent accompanied by a player-selected human who chats with the player. Materials and Methods: Sixteen participants (3 women, 24.4 ± 7.7 years old) played a competitive arm exercise game in the above four conditions. Exercise intensity was measured with inertial sensors, and four motivation scales were measured with the Intrinsic Motivation Inventory. After playing, participants answered several questions regarding their preferences. Results: The human opponent was the favorite for 14 of 16 participants and resulted in the highest interest/enjoyment and exercise intensity. All participants preferred the human opponent over the computer opponent accompanied by a human companion. Finally, 12 of 16 participants preferred the "human-like" computer opponent over the simple one. Conclusion: Our results have two implications for competitive exercise games. First, they indicate that developing computer-controlled opponents with more human-like behavior is worthwhile, but that the best results are achieved with human opponents. Second, social interaction without in-game interaction does not provide an enjoyable, intense experience. However, our results should be verified with different target populations for exercise games.

A Brief Measure of Interpersonal Interaction for 2-Player Serious Games: Questionnaire Validation

Background

Competitive and cooperative serious games have become increasingly popular in areas such as rehabilitation and education and have several potential advantages over single-player games. However, they are not suitable for everyone, and the user experience in competitive and cooperative serious games depends on many factors. One important factor is the verbal interaction between players, but the effect of this factor has not been extensively studied because of the lack of a validated measurement tool.

Objective

This paper aimed to validate a brief questionnaire that measures the verbal interaction between 2 players of a serious game. The questionnaire consists of 8 questions pertaining to the amount of conversation, its valence (positive or negative emotion), and its game relatedness.

Methods

The questionnaire was validated with 30 pairs of participants who played a competitive serious game for 10 min while being recorded with cameras. The questionnaire was filled out by both participants, an in-person observer, and 2 members of our research group who watched the videos. Results from these raters were used to develop questionnaire instructions, and the finalized questionnaire was given to 2 additional raters who were trained on 5 videos and then rated the other 25 videos independently.

Results

The questionnaire’s interrater reliability is excellent for the amount of conversation and its game relatedness (intraclass correlation coefficients [ICCs] above 0.9). Interrater reliability is fair to good for conversation valence (ICCs between 0.4 and 0.7). We believe that the lower interrater reliability for valence is primarily because of a limited spread of valence values in our sample. Furthermore, questionnaire ratings were significantly correlated with players’ personality characteristics (eg, amount of conversation was correlated with extraversion) and pressure/tension experienced in the competitive game.

Conclusions

The validated questionnaire has the potential to be a useful tool for studying user experience in competitive and cooperative serious games. Furthermore, it could be adapted for other applications such as entertainment games. However, it has only been validated with unimpaired university students in a 2-player competitive serious game and should next be validated with different target populations (eg, stroke survivors) and different game designs (eg, cooperative games).

The "Beam-Me-In Strategy" - remote haptic therapist-patient interaction with two exoskeletons for stroke therapy

Background

We present a robot-assisted telerehabilitation system that allows for haptic interaction between therapist and patient over distance. It consists of two arm therapy robots. Attached to one robot the therapists can feel on their own arm the limitations of the patient’s arm which is attached to the other robot. Due to the exoskeleton structure of the robot, movements can be performed in the three-dimensional space.

Methods

Fifteen physical and occupational therapists tested this strategy, named “Beam-Me-In”, while using an exoskeleton robot connected to a second exoskeleton robot in the same room used by the study experimenter. Furthermore, the therapists assessed the level of impairment of recorded and simulated arm movements. They quantified four typical impairments of stroke patients: reduced range of motion (active and passive), resistance to passive movement, a lack of ability to fractionate a movement, and disturbed quality of movement.

Results

On a Likert Scale (0 to 5 points) therapists rated the “Beam-Me-In” strategy as a very useful medium (mode: 4 points) to evaluate a patient’s progress over time. The passive range of motion of the elbow joint was assessed with a mean absolute error of 4.9^∘ (absolute precision error: 6.4^∘). The active range of motion of the elbow was assessed with a mean absolute error of 4.9^∘ (absolute precision error: 6.5^∘). The resistance to passive movement (i.e. modified Tardieu Scale) and the lack of ability to fractionate a movement (i.e. quantification of pathological muscle synergies) was assessed with an inter-rater reliability of 0.930 and 0.948, respectively.

Conclusions

The “Beam-Me-In” strategy is a promising approach to complement robot-assisted movement training. It can serve as a platform to assess and identify abnormal movement patterns in patients. This is the first application of remote three-dimensional haptic assessmen t applied to telerehabilitation. Furthermore, the “Beam-Me-In” strategy has a potential to overcome barriers for therapists regarding robot-assisted telerehabilitation.

Trends in robot-assisted and virtual reality-assisted neuromuscular therapy: a systematic review of health-related multiplayer games

BACKGROUND

Multiplayer games have emerged as a promising approach to increase the motivation of patients involved in rehabilitation therapy. In this systematic review, we evaluated recent publications in health-related multiplayer games that involved patients with cognitive and/or motor impairments. The aim was to investigate the effect of multiplayer gaming on game experience and game performance in healthy and non-healthy populations in comparison to individual game play. We further discuss the publications within the context of the theory of flow and the challenge point framework.

METHODS

A systematic search was conducted through EMBASE, Medline, PubMed, Cochrane, CINAHL and PsycINFO. The search was complemented by recent publications in robot-assisted multiplayer neurorehabilitation. The search was restricted to robot-assisted or virtual reality-based training.

RESULTS

Thirteen articles met the inclusion criteria. Multiplayer modes used in health-related multiplayer games were: competitive, collaborative and co-active multiplayer modes. Multiplayer modes positively affected game experience in nine studies and game performance in six studies. Two articles reported increased game performance in single-player mode when compared to multiplayer mode.

CONCLUSIONS

The multiplayer modes of training reviewed improved game experience and game performance compared to single-player modes. However, the methods reviewed were quite heterogeneous and not exhaustive. One important take-away is that adaptation of the game conditions can individualize the difficulty of a game to a player's skill level in competitive multiplayer games. Robotic assistance and virtual reality can enhance individualization by, for example, adapting the haptic conditions, e.g. by increasing haptic support or by providing haptic resistance. The flow theory and the challenge point framework support these results and are used in this review to frame the idea of adapting players' game conditions.

Music meets robotics: a prospective randomized study on motivation during robot aided therapy

BACKGROUND

Robots have been successfully applied in motor training during neurorehabilitation. As music is known to improve motor function and motivation in neurorehabilitation training, we aimed at integrating music creation into robotic-assisted motor therapy. We developed a virtual game-like environment with music for the arm therapy robot ARMin, containing four different motion training conditions: a condition promoting creativity (C+) and one not promoting creativity (C-), each in a condition with (V+) and without (V-) a visual display (i.e., a monitor). The visual display was presenting the game workspace but not contributing to the creative process itself. In all four conditions the therapy robot haptically displayed the game workspace. Our aim was to asses the effects of creativity and visual display on motivation.

METHODS

In a prospective randomized single-center study, healthy participants were randomly assigned to play two of the four training conditions, either with (V+) or without visual display (V-). In the third round, the participants played a repetition of the preferred condition of the two first rounds, this time with a new V condition (i.e., with or without visual display). For each of the three rounds, motivation was measured with the Intrinsic Motivation Inventory (IMI) in the subscales interest/enjoyment, perceived choice, value/usefulness, and man-machine-relation. We recorded the actual training time, the time of free movement, and the velocity profile and administered a questionnaire to measure perceived training time and perceived effort. All measures were analysed using linear mixed models. Furthermore, we asked if the participants would like to receive the created music piece.

RESULTS

Sixteen healthy subjects (ten males, six females, mean age: 27.2 years, standard deviation: 4.1 years) with no known motor or cognitive deficit participated. Promotion of creativity (i.e., C+ instead of C-) significantly increased the IMI-item interest/enjoyment (p=0.001) and the IMI-item perceived choice (p=0.010). We found no significant effects in the IMI-items man-machine relation and value/usefulness. Conditions promoting creativity (with or without visual display) were preferred compared to the ones not promoting creativity. An interaction effect of promotion of creativity and omission of visual display was present for training time (p=0.013) and training intensity (p<0.001). No differences in relative perceived training time, perceived effort, and perceived value among the four training conditions were found.

CONCLUSIONS

Promoting creativity in a visuo-audio-haptic or audio-haptic environment increases motivation in robot-assisted therapy. We demonstrated the feasibility of performing an audio-haptic music creation task and recommend to try the system on patients with neuromuscular disorders.

TRIAL REGISTRATION

ClinicalTrials.gov, NCT02720341. Registered 25 March 2016, https://clinicaltrials.gov/ct2/show/NCT02720341.

Comparison of two difficulty adaptation strategies for competitive arm rehabilitation exercises

This paper presents two different strategies for difficulty adaptation in a competitive arm rehabilitation game: a manual adaptation strategy and an automatic performance-based adaptation strategy. The two strategies were implemented in a competitive game controlled with an inertial-sensor-based home rehabilitation device. They were first evaluated with 32 pairs of unimpaired participants, who played the game with manual adaptation, automated adaptation, or no adaptation. Each variant was played for 9 minutes. Then, the manual and automatic adaptation were also tested by 5 pairs consisting of a person with arm impairment (due to neurological injury) and their unimpaired friend or relative. Throughout the game, motivation was measured with questionnaires while exercise intensity was tracked using the inertial sensors. Results showed that both manual and automatic difficulty adaptation lead to higher motivation and exercise intensity than no adaptation. Unimpaired participants showed no clear preference between manual and automatic adaptation while 4 of 5 impaired participants preferred automatic adaptation. For future use, we propose a combination of manual and automatic adaptation that should be evaluated with more impaired participants in longer multisession experiments.

A multisession evaluation of an adaptive competitive arm rehabilitation game

BACKGROUND

People with neurological injuries such as stroke should exercise frequently and intensely to regain their motor abilities, but are generally hindered by lack of motivation. One way to increase motivation in rehabilitation is through competitive exercises, but such exercises have only been tested in single brief sessions and usually did not adapt difficulty to the patient's abilities.

METHODS

We designed a competitive arm rehabilitation game for two players that dynamically adapts its difficulty to both players' abilities. This game was evaluated by two participant groups: 15 participants with chronic arm impairment who exercised at home with an unimpaired friend or relative, and 20 participants in the acute or subacute phase of stroke who exercised in pairs (10 pairs) at a rehabilitation clinic. All participants first played the game against their human opponent for 3 sessions, then played alone (against a computer opponent) in the final, fourth session. In all sessions, participants' subjective experiences were assessed with the Intrinsic Motivation Inventory questionnaire while exercise intensity was measured using inertial sensors built into the rehabilitation device. After the fourth session, a final brief questionnaire was used to compare competition and exercising alone.

RESULTS

Participants who played against an unimpaired friend or relative at home tended to prefer competition (only 1 preferred exercising alone), and exhibited higher enjoyment and exercise intensity when competing (first three sessions) than when exercising alone (last session). Participants who played against each other in the clinic, however, did not exhibit significant differences between competition and exercising alone. For both groups, there was no difference in enjoyment or exercise intensity between the first three sessions, indicating no negative effects of habituation or novelty.

CONCLUSIONS

Competitive exercises have high potential for unsupervised home rehabilitation, as they improve enjoyment and exercise intensity compared to exercising alone. Such exercises could thus improve rehabilitation outcome, but this needs to be tested in long-term clinical trials. It is not clear why participants who competed against each other at the clinic did not exhibit any advantages of competition, and further studies are needed to determine how different factors (environment, nature of opponent etc.) influence patients' experiences with competitive exercises.

TRIAL REGISTRATION

The study is not a clinical trial. While human subjects are involved, they do not participate in a full rehabilitation intervention, and no health outcomes are examined.

Balancing the playing field: collaborative gaming for physical training

Background

Multiplayer video games promoting exercise-based rehabilitation may facilitate motor learning, by increasing motivation through social interaction. However, a major design challenge is to enable meaningful inter-subject interaction, whilst allowing for significant skill differences between players. We present a novel motor-training paradigm that allows real-time collaboration and performance enhancement, across a wide range of inter-subject skill mismatches, including disabled vs. able-bodied partnerships.

Methods

A virtual task consisting of a dynamic ball on a beam, is controlled at each end using independent digital force-sensing handgrips. Interaction is mediated through simulated physical coupling and locally-redundant control. Game performance was measured in 16 healthy-healthy and 16 patient-expert dyads, where patients were hemiparetic stroke survivors using their impaired arm. Dual-player was compared to single-player performance, in terms of score, target tracking, stability, effort and smoothness; and questionnaires probing user-experience and engagement.

Results

Performance of less-able subjects (as ranked from single-player ability) was enhanced by dual-player mode, by an amount proportionate to the partnership’s mismatch. The more abled partners’ performances decreased by a similar amount. Such zero-sum interactions were observed for both healthy-healthy and patient-expert interactions. Dual-player was preferred by the majority of players independent of baseline ability and subject group; healthy subjects also felt more challenged, and patients more skilled.

Conclusion

This is the first demonstration of implicit skill balancing in a truly collaborative virtual training task leading to heightened engagement, across both healthy subjects and stroke patients.

Electronic supplementary material

The online version of this article (doi:10.1186/s12984-017-0319-x) contains supplementary material, which is available to authorized users.