Towards the Design of a Ring Sensor-based mHealth System to Achieve Optimal Motor Function in Stroke Survivors

Perspectives and expectations of stroke survivors using egocentric cameras for monitoring hand function at home: a mixed methods study

PURPOSE

Most stroke survivors have remaining upper limb impairment six months after stroke and require additional rehabilitation and help from family members to enhance their performance of daily activities. First-person (egocentric) video has been proposed to capture the activities of daily living (ADLs) of stroke survivors in order to assess their hand function at home. This study explored the experiences and expectations of stroke survivors regarding the use of egocentric cameras in daily life for rehabilitation applications.

METHODS

Twenty-one chronic stroke survivors recruited for the study were asked to record three sessions of 1.5 h of video of their ADLs at home over two weeks. Their experiences and expectations after completing the recordings were discussed using a structured questionnaire and a semi-structured interview. The questionnaire and interview data were analysed using descriptive statistics and content analysis, respectively. The results were further integrated using a mixed methods analysis for mutual explanation and elaboration.

RESULTS

The themes generated were Camera Usability, Privacy Concerns Related to Home Recordings, Future Use of the Camera in Public, and Information Usefulness. The participants perceived that the camera was easy to use, the information obtained from the recordings was beneficial, and no major concerns about recording at home. A discreet camera and a solution to privacy issues were prerequisites to recording tasks in public.

CONCLUSIONS

There was high acceptance among stroke survivors regarding the use of wearable cameras for rehabilitation purposes in the future. Concerns to be managed include discomfort, self-consciousness, and the privacy of others.Implications for rehabilitationThe egocentric camera was easy for the stroke survivors to use at home. However, they expressed a preference for cameras to be less noticeable and lighter in the future to minimize self-consciousness and discomfort.Expectations for future use of an egocentric camera for upper limb rehabilitation at home from the perspectives of stroke survivors included receiving feedback on their hand function in daily life and guidance on how to improve function.Privacy concerns of stroke survivors regarding recording activities of daily living were mostly avoidable by planning in advance. However, some personal hygiene tasks and virtual meetings were recorded by accident. A checklist of common activities that may raise privacy issues can be provided along with the camera to serve as a reminder to avoid these issues.

Recognizing hand use and hand role at home after stroke from egocentric video

Hand function is a central determinant of independence after stroke. Measuring hand use in the home environment is necessary to evaluate the impact of new interventions, and calls for novel wearable technologies. Egocentric video can capture hand-object interactions in context, as well as show how more-affected hands are used during bilateral tasks (for stabilization or manipulation). Automated methods are required to extract this information. The objective of this study was to use artificial intelligence-based computer vision to classify hand use and hand role from egocentric videos recorded at home after stroke. Twenty-one stroke survivors participated in the study. A random forest classifier, a SlowFast neural network, and the Hand Object Detector neural network were applied to identify hand use and hand role at home. Leave-One-Subject-Out-Cross-Validation (LOSOCV) was used to evaluate the performance of the three models. Between-group differences of the models were calculated based on the Mathews correlation coefficient (MCC). For hand use detection, the Hand Object Detector had significantly higher performance than the other models. The macro average MCCs using this model in the LOSOCV were 0.50 ± 0.23 for the more-affected hands and 0.58 ± 0.18 for the less-affected hands. Hand role classification had macro average MCCs in the LOSOCV that were close to zero for all models. Using egocentric video to capture the hand use of stroke survivors at home is technically feasible. Pose estimation to track finger movements may be beneficial to classifying hand roles in the future.

Validity of Novel Outcome Measures for Hand Function Performance After Stroke Using Egocentric Video

BACKGROUND

Evaluating upper limb (UL) interventions after stroke calls for outcome measures that describe impact on daily life in the community. UL use ratio has been used to quantify the performance domain of UL function, but generally focuses on arm use only. A hand use ratio could provide additional information about UL function after stroke. Additionally, a ratio based on the role of the more-affected hand in bilateral activities (stabilizer or manipulator) may also reflect hand function recovery. Egocentric video is a novel modality that can record both dynamic and static hand use and hand roles at home after stroke.

OBJECTIVE

To validate hand use and hand role ratios from egocentric video against standardized clinical UL assessments.

METHODS

Twenty-four stroke survivors recorded daily tasks in a home simulation laboratory and their daily routines at home using egocentric cameras. Spearman's correlation was used to compare the ratios with the Fugl-Meyer Assessment-Upper Extremity (FMA-UE), Action Research Arm Test (ARAT), and Motor Activity Log-30 (MAL, Amount of Use (AoU), and Quality of Movement (QoM)).

RESULTS

Hand use ratio significantly correlated with the FMA-UE (0.60, 95% CI: 0.26, 0.81), ARAT (0.44, CI: 0.04, 0.72), MAL-AoU (0.80, CI: 0.59, 0.91), and MAL-QoM (0.79, CI: 0.57, 0.91). Hand role ratio had no significant correlations with the assessments.

CONCLUSION

Hand use ratio automatically extracted from egocentric video, but not hand role ratio, was found to be a valid measure of hand function performance in our sample. Further investigation is necessary to interpret hand role information.

Envisioning the use of in-situ arm movement data in stroke rehabilitation: Stroke survivors' and occupational therapists' perspectives

BACKGROUND

The key for successful stroke upper-limb rehabilitation includes the personalization of therapeutic interventions based on patients' functional ability and performance level. However, therapists often encounter challenges in supporting personalized rehabilitation due to the lack of information about how stroke survivors use their stroke-affected arm outside the clinic. Wearable technologies have been considered as an effective, objective solution to monitor patients' arm use patterns in their naturalistic environments. However, these technologies have remained a proof of concept and have not been adopted as mainstream therapeutic products, and we lack understanding of how key stakeholders perceive the use of wearable technologies in their practice.

OBJECTIVE

We aim to understand how stroke survivors and therapists perceive and envision the use of wearable sensors and arm activity data in practical settings and how we could design a wearable-based performance monitoring system to better support the needs of the stakeholders.

METHODS

We conducted semi-structured interviews with four stroke survivors and 15 occupational therapists (OTs) based on real-world arm use data that we collected for contextualization. To situate our participants, we leveraged a pair of finger-worn accelerometers to collect stroke survivors' arm use data in real-world settings, which we used to create study probes for stroke survivors and OTs, respectively. The interview data was analyzed using the thematic approach.

RESULTS

Our study unveiled a detailed account of (1) the receptiveness of stroke survivors and OTs for using wearable sensors in clinical practice, (2) OTs' envisioned strategies to utilize patient-generated sensor data in the light of providing patients with personalized therapy programs, and (3) practical challenges and design considerations to address for the accelerated integration of wearable systems into their practice.

CONCLUSIONS

These findings offer promising directions for the design of a wearable solution that supports OTs to develop individually-tailored therapy programs for stroke survivors to improve their affected arm use.

Classification of functional and non-functional arm use by inertial measurement units in individuals with upper limb impairment after stroke

Background: Arm use metrics derived from wrist-mounted movement sensors are widely used to quantify the upper limb performance in real-life conditions of individuals with stroke throughout motor recovery. The calculation of real-world use metrics, such as arm use duration and laterality preferences, relies on accurately identifying functional movements. Hence, classifying upper limb activity into functional and non-functional classes is paramount. Acceleration thresholds are conventionally used to distinguish these classes. However, these methods are challenged by the high inter and intra-individual variability of movement patterns. In this study, we developed and validated a machine learning classifier for this task and compared it to methods using conventional and optimal thresholds. Methods: Individuals after stroke were video-recorded in their home environment performing semi-naturalistic daily tasks while wearing wrist-mounted inertial measurement units. Data were labeled frame-by-frame following the Taxonomy of Functional Upper Limb Motion definitions, excluding whole-body movements, and sequenced into 1-s epochs. Actigraph counts were computed, and an optimal threshold for functional movement was determined by receiver operating characteristic curve analyses on group and individual levels. A logistic regression classifier was trained on the same labels using time and frequency domain features. Performance measures were compared between all classification methods. Results: Video data (6.5 h) of 14 individuals with mild-to-severe upper limb impairment were labeled. Optimal activity count thresholds were ≥20.1 for the affected side and ≥38.6 for the unaffected side and showed high predictive power with an area under the curve (95% CI) of 0.88 (0.87,0.89) and 0.86 (0.85, 0.87), respectively. A classification accuracy of around 80% was equivalent to the optimal threshold and machine learning methods and outperformed the conventional threshold by ∼10%. Optimal thresholds and machine learning methods showed superior specificity (75-82%) to conventional thresholds (58-66%) across unilateral and bilateral activities. Conclusion: This work compares the validity of methods classifying stroke survivors' real-life arm activities measured by wrist-worn sensors excluding whole-body movements. The determined optimal thresholds and machine learning classifiers achieved an equivalent accuracy and higher specificity than conventional thresholds. Our open-sourced classifier or optimal thresholds should be used to specify the intensity and duration of arm use.

A Systematic Review of Wearable Sensors for Monitoring Physical Activity

This article reviews the use of wearable sensors for the monitoring of physical activity (PA) for different purposes, including assessment of gait and balance, prevention and/or detection of falls, recognition of various PAs, conduction and assessment of rehabilitation exercises and monitoring of neurological disease progression. The article provides in-depth information on the retrieved articles and discusses study shortcomings related to demographic factors, i.e., age, gender, healthy participants vs patients, and study conditions. It is well known that motion patterns change with age and the onset of illnesses, and that the risk of falling increases with age. Yet, studies including older persons are rare. Gender distribution was not even provided in several studies, and others included only, or a majority of, men. Another shortcoming is that none of the studies were conducted in real-life conditions. Hence, there is still important work to be done in order to increase the usefulness of wearable sensors in these areas. The article highlights flaws in how studies based on previously collected datasets report on study samples and the data collected, which makes the validity and generalizability of those studies low. Exceptions exist, such as the promising recently reported open dataset FallAllD, wherein a longitudinal study with older adults is ongoing.

An Overview of Sensors, Design and Healthcare Challenges in Smart Homes: Future Design Questions

The ageing population increases the demand for customized home care. As a result, sensing technologies are finding their way into the home environment. However, challenges associated with how users interact with sensors and data are not well-researched, particularly from a design perspective. This review explores the literature on important research projects around sensors, design and smart healthcare in smart homes, and highlights challenges for design research. A PRISMA protocol-based screening procedure is adopted to identify relevant articles (n = 180) on the subject of sensors, design and smart healthcare. The exploration and analysis of papers are performed using hierarchical charts, force-directed layouts and 'bedraggled daisy' Venn diagrams. The results show that much work has been carried out in developing sensors for smart home care. Less attention is focused on addressing challenges posed by sensors in homes, such as data accessibility, privacy, comfort, security and accuracy, and how design research might solve these challenges. This review raises key design research questions, particularly in working with sensors in smart home environments.

Use of artificial intelligence as an instrument of evaluation after stroke: a scoping review based on international classification of functioning, disability and health concept

Introduction: To understand the current practices in stroke evaluation, the main clinical decision support system and artificial intelligence (AI) technologies need to be understood to assist the therapist in obtaining better insights about impairments and level of activity and participation in persons with stroke during rehabilitation. Methods: This scoping review maps the use of AI for the functional evaluation of persons with stroke; the context involves any setting of rehabilitation. Data were extracted from CENTRAL, MEDLINE, EMBASE, LILACS, CINAHL, PEDRO Web of Science, IEEE Xplore, AAAI Publications, ACM Digital Library, MathSciNet, and arXiv up to January 2021. The data obtained from the literature review were summarized in a single dataset in which each reference paper was considered as an instance, and the study characteristics were considered as attributes. The attributes used for the multiple correspondence analysis were publication year, study type, sample size, age, stroke phase, stroke type, functional status, AI type, and AI function. Results: Forty-four studies were included. The analysis showed that spasticity analysis based on ML techniques was used for the cases of stroke with moderate functional status. The techniques of deep learning and pressure sensors were used for gait analysis. Machine learning techniques and algorithms were used for upper limb and reaching analyses. The inertial measurement unit technique was applied in studies where the functional status was between mild and severe. The fuzzy logic technique was used for activity classifiers. Conclusion: The prevailing research themes demonstrated the growing utility of AI algorithms for stroke evaluation.