Evaluation of a smartphone human activity recognition application with able-bodied and stroke participants

Validity of an android device for assessing mobility in people with chronic stroke and hemiparesis: a cross-sectional study

BACKGROUND

Incorporating instrument measurements into clinical assessments can improve the accuracy of results when assessing mobility related to activities of daily living. This can assist clinicians in making evidence-based decisions. In this context, kinematic measures are considered essential for the assessment of sensorimotor recovery after stroke. The aim of this study was to assess the validity of using an Android device to evaluate kinematic data during the performance of a standardized mobility test in people with chronic stroke and hemiparesis.

METHODS

This is a cross-sectional study including 36 individuals with chronic stroke and hemiparesis and 33 age-matched healthy subjects. A simple smartphone attached to the lumbar spine with an elastic band was used to measure participants' kinematics during a standardized mobility test by using the inertial sensor embedded in it. This test includes postural control, walking, turning and sitting down, and standing up. Differences between stroke and non-stroke participants in the kinematic parameters obtained after data sensor processing were studied, as well as in the total execution and reaction times. Also, the relationship between the kinematic parameters and the community ambulation ability, degree of disability and functional mobility of individuals with stroke was studied.

RESULTS

Compared to controls, participants with chronic stroke showed a larger medial-lateral displacement (p = 0.022) in bipedal stance, a higher medial-lateral range (p < 0.001) and a lower cranio-caudal range (p = 0.024) when walking, and lower turn-to-sit power (p = 0.001), turn-to-sit jerk (p = 0.026) and sit-to-stand jerk (p = 0.001) when assessing turn-to-sit-to-stand. Medial-lateral range and total execution time significantly correlated with all the clinical tests (p < 0.005), and resulted significantly different between independent and limited community ambulation patients (p = 0.042 and p = 0.006, respectively) as well as stroke participants with significant disability or slight/moderate disability (p = 0.024 and p = 0.041, respectively).

CONCLUSION

This study reports a valid, single, quick and easy-to-use test for assessing kinematic parameters in chronic stroke survivors by using a standardized mobility test with a smartphone. This measurement could provide valid clinical information on reaction time and kinematic parameters of postural control and gait, which can help in planning better intervention approaches.

Smartphone-based gait and balance assessment in survivors of stroke: a systematic review

PURPOSE

Gait and balance impairments are associated with falls and reduced quality of life among survivors of stroke (SS). Effective methods to assess these impairments at-home and in-clinic can help reduce fall risks and improve functional outcomes. Smartphone technology may be able to evaluate these impairments. This review aims to summarize the validity, reliability, sensitivity, and specificity of smartphone applications for determining gait and balance disorders in SS.

METHOD

Database search through PubMed, Web of Science, Scopus, CINAHL, and SportDiscuss was conducted to retrieve studies that explored the use of smartphone-based applications for assessing gait and balance disorders in SS. Two independent reviewers screened potential articles to determine eligibility for inclusion. Eligible studies were summarized for participant and study characteristics, validity, reliability, sensitivity, and specificity of smartphone assessments. Methodological quality assessment of studies was performed using the NIH Quality Assessment Tool.

RESULTS

Seven cross-sectional studies were included in the review. Quality assessment revealed all studies had low risk of bias. Three of the included studies examined the validity, four examined the reliability, and two examined the specificity and sensitivity of smartphone-based application assessments of gait and balance in SS. Studies revealed that smartphones were valid, reliable, specific, and sensitive. Six of the seven included studies intended their use for SS and one study for clinicians.

CONCLUSION

Preliminary evidence supports that smartphone-based gait and balance assessments are valid, reliable, sensitive, and specific in SS in laboratory settings. Future research is needed to test smartphone-based gait and balance assessments in home settings and determine optimal wear sites for assessments.IMPLICATIONS FOR REHABILITATIONSmartphone-based gait and balance assessments are feasible, valid and reliable for survivors of strokeThe findings may guide future research to standardize the use of smartphone to assess gait and balance in this population.The remote use of smartphone-based assessments to predict fall risk in survivors of stroke needs to be explored.

Designing an Egocentric Video-Based Dashboard to Report Hand Performance Measures for Outpatient Rehabilitation of Cervical Spinal Cord Injury

Background

Functional use of the upper extremities (UEs) is a top recovery priority for individuals with cervical spinal cord injury (cSCI), but the inability to monitor recovery at home and limitations in hand function outcome measures impede optimal recovery.

Objectives

We developed a framework using wearable cameras to monitor hand use at home and aimed to identify the best way to report information to clinicians.

Methods

A dashboard was iteratively developed with clinician (n = 7) input through focus groups and interviews, creating low-fidelity prototypes based on recurring feedback until no new information emerged. Affinity diagramming was used to identify themes and subthemes from interview data. User stories were developed and mapped to specific features to create a high-fidelity prototype.

Results

Useful elements identified for a dashboard reporting hand performance included summaries to interpret graphs, a breakdown of hand posture and activity to provide context, video snippets to qualitatively view hand use at home, patient notes to understand patient satisfaction or struggles, and time series graphing of metrics to measure trends over time.

Conclusion

Involving end-users in the design process and breaking down user requirements into user stories helped identify necessary interface elements for reporting hand performance metrics to clinicians. Clinicians recognized the dashboard's potential to monitor rehabilitation progress, provide feedback on hand use, and track progress over time. Concerns were raised about the implementation into clinical practice, therefore further inquiry is needed to determine the tool's feasibility and usefulness in clinical practice for individuals with UE impairments.

Patient experience using digital therapy "Vigo" for stroke patient recovery: a qualitative descriptive study

BACKGROUND

The digital assistant "Vigo" is a computer-generated artificial intelligence-based application that serves as a digital assistant to a stroke patient and his family. With its conversational chatbot and gamification elements it counsels, educates, and trains the stroke patient and patient's family on stroke, rehabilitation, care, and other related issues.

AIM

This study describes insights about The digital assitant "Vigo" usability from a patients' perspective.

METHODS

Twelve patients tested the application at their home environment. Three semi-structured interviews were conducted with each participant to obtain information on the usability of the application. Deductive thematic analyses were used to analyze trancripts.

RESULTS

Participants expressed their opinions on music, pictures, video and audio files, chat options, layout, text, name of application and stand that is used for placement of devices on which "Vigo" is installed on. All participants generally evaluated application as transparent, understandable, and handy. The overall design of the application was rated as good. Participants were mostly unsatisfied with difficulty level and diversity of exercises.

CONCLUSIONS

Participants had a positive attitude towards using tablet tehchnologies in their home environment. Users of digital assistant "Vigo" acknowledged its ability to support, give educational information and increase participation in therapeutic activities.Implications for rehabilitationTablet application can support, give educational information, and increase participation in therapeutic activities for persons after stroke.As home-based rehabilitation tool, the content of the application must be simple, flexible, and diverse, to face the challenges of meeting each individual's goals, functional needs and abilities.

A scoping review on recent trends in wearable sensors to analyze gait in people with stroke: From sensor placement to validation against gold-standard equipment

The purpose of the review is to evaluate wearable sensor placement, their impact and validation of wearable sensors on analyzing gait, primarily the postural instability in people with stroke. Databases, namely PubMed, Cochrane, SpringerLink, and IEEE Xplore were searched to identify related articles published since January 2005. The authors have selected the articles by considering patient characteristics, intervention details, and outcome measurements by following the priorly set inclusion and exclusion criteria. From a total of 1077 articles, 142 were included in this study and classified into functional fields, namely postural stability (PS) assessments, physical activity monitoring (PA), gait pattern classification (GPC), and foot drop correction (FDC). The review covers the types of wearable sensors, their placement, and their performance in terms of reliability and validity. When employing a single wearable sensor, the pelvis and foot were the most used locations for detecting gait asymmetry and kinetic parameters, respectively. Multiple Inertial Measurement Units placed at different body parts were effectively used to estimate postural stability and gait pattern. This review article has compared results of placement of sensors at different locations helping researchers and clinicians to identify the best possible placement for sensors to measure specific kinematic and kinetic parameters in persons with stroke.

Improving Inertial Sensor-Based Activity Recognition in Neurological Populations

Inertial sensor-based human activity recognition (HAR) has a range of healthcare applications as it can indicate the overall health status or functional capabilities of people with impaired mobility. Typically, artificial intelligence models achieve high recognition accuracies when trained with rich and diverse inertial datasets. However, obtaining such datasets may not be feasible in neurological populations due to, e.g., impaired patient mobility to perform many daily activities. This study proposes a novel framework to overcome the challenge of creating rich and diverse datasets for HAR in neurological populations. The framework produces images from numerical inertial time-series data (initial state) and then artificially augments the number of produced images (enhanced state) to achieve a larger dataset. Here, we used convolutional neural network (CNN) architectures by utilizing image input. In addition, CNN enables transfer learning which enables limited datasets to benefit from models that are trained with big data. Initially, two benchmarked public datasets were used to verify the framework. Afterward, the approach was tested in limited local datasets of healthy subjects (HS), Parkinson's disease (PD) population, and stroke survivors (SS) to further investigate validity. The experimental results show that when data augmentation is applied, recognition accuracies have been increased in HS, SS, and PD by 25.6%, 21.4%, and 5.8%, respectively, compared to the no data augmentation state. In addition, data augmentation contributes to better detection of stair ascent and stair descent by 39.1% and 18.0%, respectively, in limited local datasets. Findings also suggest that CNN architectures that have a small number of deep layers can achieve high accuracy. The implication of this study has the potential to reduce the burden on participants and researchers where limited datasets are accrued.

Using Sensor Technology to Measure Gait Capacity and Gait Performance in Rehabilitation Inpatients with Neurological Disorders

The aim of this study was to objectively assess and compare gait capacity and gait performance in rehabilitation inpatients with stroke or incomplete spinal cord injury (iSCI) using inertial measurement units (IMUs). We investigated how gait capacity (what someone can do) is related to gait performance (what someone does). Twenty-two inpatients (11 strokes, 11 iSCI) wore ankle positioned IMUs during the daytime to assess gait. Participants completed two circuits to assess gait capacity. These were videotaped to certify the validity of the IMU algorithm. Regression analyses were used to investigate if gait capacity was associated with gait performance (i.e., walking activity and spontaneous gait characteristics beyond therapy time). The ankle positioned IMUs validly assessed the number of steps, walking time, gait speed, and stride length (r ≥ 0.81). The walking activity was strongly (r ≥ 0.76) related to capacity-based gait speed. Maximum spontaneous gait speed and stride length were similar to gait capacity. However, the average spontaneous gait speed was half the capacity-based gait speed. Gait capacity can validly be assessed using IMUs and is strongly related to gait performance in rehabilitation inpatients with neurological disorders. Measuring gait performance with IMUs provides valuable additional information about walking activity and spontaneous gait characteristics to inform about functional recovery.

Accuracy of gait and posture classification using movement sensors in individuals with mobility impairment after stroke

Background: Stroke leads to motor impairment which reduces physical activity, negatively affects social participation, and increases the risk of secondary cardiovascular events. Continuous monitoring of physical activity with motion sensors is promising to allow the prescription of tailored treatments in a timely manner. Accurate classification of gait activities and body posture is necessary to extract actionable information for outcome measures from unstructured motion data. We here develop and validate a solution for various sensor configurations specifically for a stroke population.

Methods: Video and movement sensor data (locations: wrists, ankles, and chest) were collected from fourteen stroke survivors with motor impairment who performed real-life activities in their home environment. Video data were labeled for five classes of gait and body postures and three classes of transitions that served as ground truth. We trained support vector machine (SVM), logistic regression (LR), and k-nearest neighbor (kNN) models to identify gait bouts only or gait and posture. Model performance was assessed by the nested leave-one-subject-out protocol and compared across five different sensor placement configurations.

Results: Our method achieved very good performance when predicting real-life gait versus non-gait (Gait classification) with an accuracy between 85% and 93% across sensor configurations, using SVM and LR modeling. On the much more challenging task of discriminating between the body postures lying, sitting, and standing as well as walking, and stair ascent/descent (Gait and postures classification), our method achieves accuracies between 80% and 86% with at least one ankle and wrist sensor attached unilaterally. The Gait and postures classification performance between SVM and LR was equivalent but superior to kNN.

Conclusion: This work presents a comparison of performance when classifying Gait and body postures in post-stroke individuals with different sensor configurations, which provide options for subsequent outcome evaluation. We achieved accurate classification of gait and postures performed in a real-life setting by individuals with a wide range of motor impairments due to stroke. This validated classifier will hopefully prove a useful resource to researchers and clinicians in the increasingly important field of digital health in the form of remote movement monitoring using motion sensors.

Brazilian practice guidelines for stroke rehabilitation: Part II

The Brazilian Practice Guidelines for Stroke Rehabilitation - Part II, developed by the Scientific Department of Neurological Rehabilitation of the Brazilian Academy of Neurology (Academia Brasileira de Neurologia, in Portuguese), focuses on specific rehabilitation techniques to aid recovery from impairment and disability after stroke. As in Part I, Part II is also based on recently available evidence from randomized controlled trials, systematic reviews, meta-analyses, and other guidelines. Part II covers disorders of communication, dysphagia, postural control and balance, ataxias, spasticity, upper limb rehabilitation, gait, cognition, unilateral spatial neglect, sensory impairments, home rehabilitation, medication adherence, palliative care, cerebrovascular events related to severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection, the future of stroke rehabilitation, and stroke websites to support patients and caregivers. Our goal is to provide health professionals with more recent knowledge and recommendations for better rehabilitation care after stroke.

Smartphone-Based Activity Recognition Using Multistream Movelets Combining Accelerometer and Gyroscope Data

Physical activity patterns can reveal information about one’s health status. Built-in sensors in a smartphone, in comparison to a patient’s self-report, can collect activity recognition data more objectively, unobtrusively, and continuously. A variety of data analysis approaches have been proposed in the literature. In this study, we applied the movelet method to classify the activities performed using smartphone accelerometer and gyroscope data, which measure a phone’s acceleration and angular velocity, respectively. The movelet method constructs a personalized dictionary for each participant using training data and classifies activities in new data with the dictionary. Our results show that this method has the advantages of being interpretable and transparent. A unique aspect of our movelet application involves extracting unique information, optimally, from multiple sensors. In comparison to single-sensor applications, our approach jointly incorporates the accelerometer and gyroscope sensors with the movelet method. Our findings show that combining data from the two sensors can result in more accurate activity recognition than using each sensor alone. In particular, the joint-sensor method reduces errors of the gyroscope-only method in differentiating between standing and sitting. It also reduces errors in the accelerometer-only method when classifying vigorous activities.

Use of artificial intelligence as an instrument of evaluation after stroke: a scoping review based on international classification of functioning, disability and health concept

Introduction: To understand the current practices in stroke evaluation, the main clinical decision support system and artificial intelligence (AI) technologies need to be understood to assist the therapist in obtaining better insights about impairments and level of activity and participation in persons with stroke during rehabilitation. Methods: This scoping review maps the use of AI for the functional evaluation of persons with stroke; the context involves any setting of rehabilitation. Data were extracted from CENTRAL, MEDLINE, EMBASE, LILACS, CINAHL, PEDRO Web of Science, IEEE Xplore, AAAI Publications, ACM Digital Library, MathSciNet, and arXiv up to January 2021. The data obtained from the literature review were summarized in a single dataset in which each reference paper was considered as an instance, and the study characteristics were considered as attributes. The attributes used for the multiple correspondence analysis were publication year, study type, sample size, age, stroke phase, stroke type, functional status, AI type, and AI function. Results: Forty-four studies were included. The analysis showed that spasticity analysis based on ML techniques was used for the cases of stroke with moderate functional status. The techniques of deep learning and pressure sensors were used for gait analysis. Machine learning techniques and algorithms were used for upper limb and reaching analyses. The inertial measurement unit technique was applied in studies where the functional status was between mild and severe. The fuzzy logic technique was used for activity classifiers. Conclusion: The prevailing research themes demonstrated the growing utility of AI algorithms for stroke evaluation.

A Blockchain-Enabled Framework for mHealth Systems

Presently modern technology makes a significant contribution to the transition from traditional healthcare to smart healthcare systems. Mobile health (mHealth) uses advances in wearable sensors, telecommunications and the Internet of Things (IoT) to propose a new healthcare concept centered on the patient. Patients' real-time remote continuous health monitoring, remote diagnosis, treatment, and therapy is possible in an mHealth system. However, major limitations include the transparency, security, and privacy of health data. One possible solution to this is the use of blockchain technologies, which have found numerous applications in the healthcare domain mainly due to theirs features such as decentralization (no central authority is needed), immutability, traceability, and transparency. We propose an mHealth system that uses a private blockchain based on the Ethereum platform, where wearable sensors can communicate with a smart device (a smartphone or smart tablet) that uses a peer-to-peer hypermedia protocol, the InterPlanetary File System (IPFS), for the distributed storage of health-related data. Smart contracts are used to create data queries, to access patient data by healthcare providers, to record diagnostic, treatment, and therapy, and to send alerts to patients and medical professionals.

Measuring Activities of Daily Living in Stroke Patients with Motion Machine Learning Algorithms: A Pilot Study

Measuring activities of daily living (ADLs) using wearable technologies may offer higher precision and granularity than the current clinical assessments for patients after stroke. This study aimed to develop and determine the accuracy of detecting different ADLs using machine-learning (ML) algorithms and wearable sensors. Eleven post-stroke patients participated in this pilot study at an ADL Simulation Lab across two study visits. We collected blocks of repeated activity (“atomic” activity) performance data to train our ML algorithms during one visit. We evaluated our ML algorithms using independent semi-naturalistic activity data collected at a separate session. We tested Decision Tree, Random Forest, Support Vector Machine (SVM), and eXtreme Gradient Boosting (XGBoost) for model development. XGBoost was the best classification model. We achieved 82% accuracy based on ten ADL tasks. With a model including seven tasks, accuracy improved to 90%. ADL tasks included chopping food, vacuuming, sweeping, spreading jam or butter, folding laundry, eating, brushing teeth, taking off/putting on a shirt, wiping a cupboard, and buttoning a shirt. Results provide preliminary evidence that ADL functioning can be predicted with adequate accuracy using wearable sensors and ML. The use of external validation (independent training and testing data sets) and semi-naturalistic testing data is a major strength of the study and a step closer to the long-term goal of ADL monitoring in real-world settings. Further investigation is needed to improve the ADL prediction accuracy, increase the number of tasks monitored, and test the model outside of a laboratory setting.

Scoping Review of Healthcare Literature on Mobile, Wearable, and Textile Sensing Technology for Continuous Monitoring

Remote monitoring of health can reduce frequent hospitalisations, diminishing the burden on the healthcare system and cost to the community. Patient monitoring helps identify symptoms associated with diseases or disease-driven disorders, which makes it an essential element of medical diagnoses, clinical interventions, and rehabilitation treatments for severe medical conditions. This monitoring can be expensive and time-consuming and provide an incomplete picture of the state of the patient. In the last decade, there has been a significant increase in the adoption of mobile and wearable devices, along with the introduction of smart textile solutions that offer the possibility of continuous monitoring. These alternatives fuel a technology shift in healthcare, one that involves the continuous tracking and monitoring of individuals. This scoping review examines how mobile, wearable, and textile sensing technology have been permeating healthcare by offering alternate solutions to challenging issues, such as personalised prescriptions or home-based secondary prevention. To do so, we have selected 222 healthcare literature articles published from 2007 to 2019 and reviewed them following the PRISMA process under the schema of a scoping review framework. Overall, our findings show a recent increase in research on mobile sensing technology to address patient monitoring, reflected by 128 articles published in journals and 19 articles in conference proceedings between 2014 and 2019, which represents 57.65% and 8.55% respectively of all included articles.

On the Challenges and Potential of Using Barometric Sensors to Track Human Activity

Barometers are among the oldest engineered sensors. Historically, they have been primarily used either as environmental sensors to measure the atmospheric pressure for weather forecasts or as altimeters for aircrafts. With the advent of microelectromechanical system (MEMS)-based barometers and their systematic embedding in smartphones and wearable devices, a vast breadth of new applications for the use of barometers has emerged. For instance, it is now possible to use barometers in conjunction with other sensors to track and identify a wide range of human activity classes. However, the effectiveness of barometers in the growing field of human activity recognition critically hinges on our understanding of the numerous factors affecting the atmospheric pressure, as well as on the properties of the sensor itself-sensitivity, accuracy, variability, etc. This review article thoroughly details all these factors and presents a comprehensive report of the numerous studies dealing with one or more of these factors in the particular framework of human activity tracking and recognition. In addition, we specifically collected some experimental data to illustrate the effects of these factors, which we observed to be in good agreement with the findings in the literature. We conclude this review with some suggestions on some possible future uses of barometric sensors for the specific purpose of tracking human activities.

Human Activity Recognition using Inertial, Physiological and Environmental Sensors: A Comprehensive Survey

In the last decade, Human Activity Recognition (HAR) has become a vibrant research area, especially due to the spread of electronic devices such as smartphones, smartwatches and video cameras present in our daily lives. In addition, the advance of deep learning and other machine learning algorithms has allowed researchers to use HAR in various domains including sports, health and well-being applications. For example, HAR is considered as one of the most promising assistive technology tools to support elderly's daily life by monitoring their cognitive and physical function through daily activities. This survey focuses on critical role of machine learning in developing HAR applications based on inertial sensors in conjunction with physiological and environmental sensors.

Reliability of the portable gait rhythmogram in post-stroke patients

BACKGROUND

Gait analysis, such as portable gait rhythmogram (PGR), provides objective information that helps in the quantitative evaluation of human locomotion.

OBJECTIVE

The purpose of this study was to assess the reliability of PGR in post-stroke patients.

METHODS

Two raters (A and B) examined 48 post-stroke patients. To assess intra-rater reliability, rater A tested subjects on three separate occasions (Days 1, 2, and 3). To assess inter-rater reliability, raters A and B independently tested participants on the same occasion (Day 3).

RESULTS

There was no significant systematic bias between test occasions or raters. Intraclass correlation coefficient values were 0.93-0.97 for intra-rater reliability at both the comfortable speed and maximum speed, and 0.97-0.98 (comfortable speed) and 0.97-0.99 (maximum speed) for inter-rater reliability. The standard error was 1.25-1.49 (comfortable speed) and 1.62-1.77 (maximum speed) for intra-rater investigation, and 1.04-1.32 (comfortable speed) and 0.91-1.26 (maximum speed) for inter-rater investigation. At the 90% confidence level, the minimum detectable change ranged from 2.9-4.1%, and the error of an individual's score at a given time point ranged from ±2.1-2.9%.

CONCLUSIONS

Based on this excellent reliability of the PGR in post-stroke patients, it can be recommended as a simple test of gait analysis in this population.

Systematic review on the application of wearable inertial sensors to quantify everyday life motor activity in people with mobility impairments

BACKGROUND

Recent advances in wearable sensor technologies enable objective and long-term monitoring of motor activities in a patient's habitual environment. People with mobility impairments require appropriate data processing algorithms that deal with their altered movement patterns and determine clinically meaningful outcome measures. Over the years, a large variety of algorithms have been published and this review provides an overview of their outcome measures, the concepts of the algorithms, the type and placement of required sensors as well as the investigated patient populations and measurement properties.

METHODS

A systematic search was conducted in MEDLINE, EMBASE, and SCOPUS in October 2019. The search strategy was designed to identify studies that (1) involved people with mobility impairments, (2) used wearable inertial sensors, (3) provided a description of the underlying algorithm, and (4) quantified an aspect of everyday life motor activity. The two review authors independently screened the search hits for eligibility and conducted the data extraction for the narrative review.

RESULTS

Ninety-five studies were included in this review. They covered a large variety of outcome measures and algorithms which can be grouped into four categories: (1) maintaining and changing a body position, (2) walking and moving, (3) moving around using a wheelchair, and (4) activities that involve the upper extremity. The validity or reproducibility of these outcomes measures was investigated in fourteen different patient populations. Most of the studies evaluated the algorithm's accuracy to detect certain activities in unlabeled raw data. The type and placement of required sensor technologies depends on the activity and outcome measure and are thoroughly described in this review. The usability of the applied sensor setups was rarely reported.

CONCLUSION

This systematic review provides a comprehensive overview of applications of wearable inertial sensors to quantify everyday life motor activity in people with mobility impairments. It summarizes the state-of-the-art, it provides quick access to the relevant literature, and it enables the identification of gaps for the evaluation of existing and the development of new algorithms.

Enhancement of Neurocognitive Assessments Using Smartphone Capabilities: Systematic Review

BACKGROUND

Comprehensive exams such as the Dean-Woodcock Neuropsychological Assessment System, the Global Deterioration Scale, and the Boston Diagnostic Aphasia Examination are the gold standard for doctors and clinicians in the preliminary assessment and monitoring of neurocognitive function in conditions such as neurodegenerative diseases and acquired brain injuries (ABIs). In recent years, there has been an increased focus on implementing these exams on mobile devices to benefit from their configurable built-in sensors, in addition to scoring, interpretation, and storage capabilities. As smartphones become more accepted in health care among both users and clinicians, the ability to use device information (eg, device position, screen interactions, and app usage) for subject monitoring also increases. Sensor-based assessments (eg, functional gait using a mobile device's accelerometer and/or gyroscope or collection of speech samples using recordings from the device's microphone) include the potential for enhanced information for diagnoses of neurological conditions; mapping the development of these conditions over time; and monitoring efficient, evidence-based rehabilitation programs.

OBJECTIVE

This paper provides an overview of neurocognitive conditions and relevant functions of interest, analysis of recent results using smartphone and/or tablet built-in sensor information for the assessment of these different neurocognitive conditions, and how human-device interactions and the assessment and monitoring of these neurocognitive functions can be enhanced for both the patient and health care provider.

METHODS

This survey presents a review of current mobile technological capabilities to enhance the assessment of various neurocognitive conditions, including both neurodegenerative diseases and ABIs. It explores how device features can be configured for assessments as well as the enhanced capability and data monitoring that will arise due to the addition of these features. It also recognizes the challenges that will be apparent with the transfer of these current assessments to mobile devices.

RESULTS

Built-in sensor information on mobile devices is found to provide information that can enhance neurocognitive assessment and monitoring across all functional categories. Configurations of positional sensors (eg, accelerometer, gyroscope, and GPS), media sensors (eg, microphone and camera), inherent sensors (eg, device timer), and participatory user-device interactions (eg, screen interactions, metadata input, app usage, and device lock and unlock) are all helpful for assessing these functions for the purposes of training, monitoring, diagnosis, or rehabilitation.

CONCLUSIONS

This survey discusses some of the many opportunities and challenges of implementing configured built-in sensors on mobile devices to enhance assessments and monitoring of neurocognitive functions as well as disease progression across neurodegenerative and acquired neurological conditions.

Feasibility of a Sensor-Based Technological Platform in Assessing Gait and Sleep of In-Hospital Stroke and Incomplete Spinal Cord Injury (iSCI) Patients

Recovery of the walking function is one of the most common rehabilitation goals of neurological patients. Sufficient and adequate sleep is a prerequisite for recovery or training. To objectively monitor patients’ progress, a combination of different sensors measuring continuously over time is needed. A sensor-based technological platform offers possibilities to monitor gait and sleep. Implementation in clinical practice is of utmost relevance and has scarcely been studied. Therefore, this study examined the feasibility of a sensor-based technological platform within the clinical setting. Participants (12 incomplete spinal cord injury (iSCI), 13 stroke) were asked to wear inertial measurement units (IMUs) around the ankles during daytime and the bed sensor was placed under their mattress for one week. Feasibility was established based on missing data, error cause, and user experience. Percentage of missing measurement days and nights was 14% and 4%, respectively. Main cause of lost measurement days was related to missing IMU sensor data. Participants were not impeded, did not experience any discomfort, and found the sensors easy to use. The sensor-based technological platform is feasible to use within the clinical rehabilitation setting for continuously monitoring gait and sleep of iSCI and stroke patients.

A Fast and Robust Deep Convolutional Neural Networks for Complex Human Activity Recognition Using Smartphone

As a significant role in healthcare and sports applications, human activity recognition (HAR) techniques are capable of monitoring humans' daily behavior. It has spurred the demand for intelligent sensors and has been giving rise to the explosive growth of wearable and mobile devices. They provide the most availability of human activity data (big data). Powerful algorithms are required to analyze these heterogeneous and high-dimension streaming data efficiently. This paper proposes a novel fast and robust deep convolutional neural network structure (FR-DCNN) for human activity recognition (HAR) using a smartphone. It enhances the effectiveness and extends the information of the collected raw data from the inertial measurement unit (IMU) sensors by integrating a series of signal processing algorithms and a signal selection module. It enables a fast computational method for building the DCNN classifier by adding a data compression module. Experimental results on the sampled 12 complex activities dataset show that the proposed FR-DCNN model is the best method for fast computation and high accuracy recognition. The FR-DCNN model only needs 0.0029 s to predict activity in an online way with 95.27% accuracy. Meanwhile, it only takes 88 s (average) to establish the DCNN classifier on the compressed dataset with less precision loss 94.18%.

Passive Sensing of Health Outcomes Through Smartphones: Systematic Review of Current Solutions and Possible Limitations

BACKGROUND

Technological advancements, together with the decrease in both price and size of a large variety of sensors, has expanded the role and capabilities of regular mobile phones, turning them into powerful yet ubiquitous monitoring systems. At present, smartphones have the potential to continuously collect information about the users, monitor their activities and behaviors in real time, and provide them with feedback and recommendations.

OBJECTIVE

This systematic review aimed to identify recent scientific studies that explored the passive use of smartphones for generating health- and well-being-related outcomes. In addition, it explores users' engagement and possible challenges in using such self-monitoring systems.

METHODS

A systematic review was conducted, following Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines, to identify recent publications that explore the use of smartphones as ubiquitous health monitoring systems. We ran reproducible search queries on PubMed, IEEE Xplore, ACM Digital Library, and Scopus online databases and aimed to find answers to the following questions: (1) What is the study focus of the selected papers? (2) What smartphone sensing technologies and data are used to gather health-related input? (3) How are the developed systems validated? and (4) What are the limitations and challenges when using such sensing systems?

RESULTS

Our bibliographic research returned 7404 unique publications. Of these, 118 met the predefined inclusion criteria, which considered publication dates from 2014 onward, English language, and relevance for the topic of this review. The selected papers highlight that smartphones are already being used in multiple health-related scenarios. Of those, physical activity (29.6%; 35/118) and mental health (27.9; 33/118) are 2 of the most studied applications. Accelerometers (57.7%; 67/118) and global positioning systems (GPS; 40.6%; 48/118) are 2 of the most used sensors in smartphones for collecting data from which the health status or well-being of its users can be inferred.

CONCLUSIONS

One relevant outcome of this systematic review is that although smartphones present many advantages for the passive monitoring of users' health and well-being, there is a lack of correlation between smartphone-generated outcomes and clinical knowledge. Moreover, user engagement and motivation are not always modeled as prerequisites, which directly affects user adherence and full validation of such systems.

Development of a gold-standard method for the identification of sedentary, light and moderate physical activities in older adults: Definitions for video annotation

OBJECTIVES

The development of a reliable method for the identification of sedentary, light and moderate physical activities in older adults. The method consists of a validated set of definitions for the identification of the initiation and termination of physical activities performed by older adult participants, video recorded during free-living and a laboratory setting.

DESIGN

Inter-rater reliability assessment in a fully crossed design.

METHODS

An iterative consensus process was used to define the initiation and termination of common activities of daily living. These definitions were then tested using videos recorded in two scenarios (1) by 9 raters who annotated a video recording, of a free-living protocol in a home environment, recorded in a first person view, using a body-worn camera and (2) by 7 raters who annotated a video recording, of older adults performing a semi-structured protocol in a living-lab environment, recorded in a third person view, using wall mounted cameras.

RESULTS

Inter-rater reliability was excellent for all items, with Krippendorff's alpha and Fleiss' kappa all above 0.84 and a percentage of agreement above 88%. All ICC(C,1) inter-rater values for the activity quantity and duration were all above 0.9.

CONCLUSIONS

This set of physical activity initiation and termination definitions offers independent researchers a gold standard method to allow for the consistent annotation of high-frequency video footage (25fps), in both a free-living and laboratory setting. When synchronised with body-worn or ambient sensors, this annotation will allow for the development and validation of physical activity classification systems to a higher resolution than before.

Relative Efficacy of Sensor Modalities for Estimating Post-Stroke Motor Impairment

Wearable inertial sensing has been beneficial in the development of measures of motor impairment after stroke. While most early work focused on the use of accelerometry, recent work has increasingly shown that rate gyroscopes may provide complementary information. Differences in performance of accelerometers and gyroscopes in activity recognition may be due to the nature of the impairment. The current approach seeks to investigate the relative sensitivity of these sensor modalities to impairment by evaluating their classification accuracy for tasks adapted from the Fugl-Meyer Assessment. Our findings indicated that, for upper-extremity motion, classifiers trained using a combination of accelerometer and rate gyroscope data performed the best (accuracy of 73.1%). Classifiers trained using accelerometer data alone and rate gyroscope data alone performed slightly worse than the combined data classifier (70.2% and 65.7%, respectively).

Machine learning in human movement biomechanics: Best practices, common pitfalls, and new opportunities

Traditional laboratory experiments, rehabilitation clinics, and wearable sensors offer biomechanists a wealth of data on healthy and pathological movement. To harness the power of these data and make research more efficient, modern machine learning techniques are starting to complement traditional statistical tools. This survey summarizes the current usage of machine learning methods in human movement biomechanics and highlights best practices that will enable critical evaluation of the literature. We carried out a PubMed/Medline database search for original research articles that used machine learning to study movement biomechanics in patients with musculoskeletal and neuromuscular diseases. Most studies that met our inclusion criteria focused on classifying pathological movement, predicting risk of developing a disease, estimating the effect of an intervention, or automatically recognizing activities to facilitate out-of-clinic patient monitoring. We found that research studies build and evaluate models inconsistently, which motivated our discussion of best practices. We provide recommendations for training and evaluating machine learning models and discuss the potential of several underutilized approaches, such as deep learning, to generate new knowledge about human movement. We believe that cross-training biomechanists in data science and a cultural shift toward sharing of data and tools are essential to maximize the impact of biomechanics research.

Wearable Movement Sensors for Rehabilitation: A Focused Review of Technological and Clinical Advances

Recent technologic advancements have enabled the creation of portable, low-cost, and unobtrusive sensors with tremendous potential to alter the clinical practice of rehabilitation. The application of wearable sensors to track movement has emerged as a promising paradigm to enhance the care provided to patients with neurologic or musculoskeletal conditions. These sensors enable quantification of motor behavior across disparate patient populations and emerging research shows their potential for identifying motor biomarkers, differentiating between restitution and compensation motor recovery mechanisms, remote monitoring, telerehabilitation, and robotics. Moreover, the big data recorded across these applications serve as a pathway to personalized and precision medicine. This article presents state-of-the-art and next-generation wearable movement sensors, ranging from inertial measurement units to soft sensors. An overview of clinical applications is presented across a wide spectrum of conditions that have potential to benefit from wearable sensors, including stroke, movement disorders, knee osteoarthritis, and running injuries. Complementary applications enabled by next-generation sensors that will enable point-of-care monitoring of neural activity and muscle dynamics during movement also are discussed.

Classification of Daily Activities for the Elderly Using Wearable Sensors

Monitoring of activities of daily living (ADL) using wearable sensors can provide an objective indication of the activity levels or restrictions experienced by patients or elderly. The current study presented a two-sensor ADL classification method designed and tested specifically with elderly subjects. Ten healthy elderly were involved in a laboratory testing with 6 types of daily activities. Two inertial measurement units were attached to the thigh and the trunk of each subject. The results indicated an overall rate of misdetection being 2.8%. The findings of the current study can be used as the first step towards a more comprehensive activity monitoring technology specifically designed for the aging population.

Activity Recognition in Individuals Walking With Assistive Devices: The Benefits of Device-Specific Models

BACKGROUND

Wearable sensors gather data that machine-learning models can convert into an identification of physical activities, a clinically relevant outcome measure. However, when individuals with disabilities upgrade to a new walking assistive device, their gait patterns can change, which could affect the accuracy of activity recognition.

OBJECTIVE

The objective of this study was to assess whether we need to train an activity recognition model with labeled data from activities performed with the new assistive device, rather than data from the original device or from healthy individuals.

METHODS

Data were collected from 11 healthy controls as well as from 11 age-matched individuals with disabilities who used a standard stance control knee-ankle-foot orthosis (KAFO), and then a computer-controlled adaptive KAFO (Ottobock C-Brace). All subjects performed a structured set of functional activities while wearing an accelerometer on their waist, and random forest classifiers were used as activity classification models. We examined both global models, which are trained on other subjects (healthy or disabled individuals), and personal models, which are trained and tested on the same subject.

RESULTS

Median accuracies of global and personal models trained with data from the new KAFO were significantly higher (61% and 76%, respectively) than those of models that use data from the original KAFO (55% and 66%, respectively) (Wilcoxon signed-rank test, P=.006 and P=.01). These models also massively outperformed a global model trained on healthy subjects, which only achieved a median accuracy of 53%. Device-specific models conferred a major advantage for activity recognition.

CONCLUSIONS

Our results suggest that when patients use a new assistive device, labeled data from activities performed with the specific device are needed for maximal precision activity recognition. Personal device-specific models yield the highest accuracy in such scenarios, whereas models trained on healthy individuals perform poorly and should not be used in patient populations.

Classification of Alzheimer's Patients through Ubiquitous Computing

Functional data analysis and artificial neural networks are the building blocks of the proposed methodology that distinguishes the movement patterns among c’s patients on different stages of the disease and classifies new patients to their appropriate stage of the disease. The movement patterns are obtained by the accelerometer device of android smartphones that the patients carry while moving freely. The proposed methodology is relevant in that it is flexible on the type of data to which it is applied. To exemplify that, it is analyzed a novel real three-dimensional functional dataset where each datum is observed in a different time domain. Not only is it observed on a difference frequency but also the domain of each datum has different length. The obtained classification success rate of 83% indicates the potential of the proposed methodology.

Intra-rater and inter-rater reliability of the portable gait rhythmogram in post-stroke patients

[Purpose] Gait analysis, such as portable gait rhythmogram (PGR) provides objective information that helps in the quantitative evaluation of human locomotion. The purpose of this study was to assess the reliability of PGR in post-stroke patients. [Subjects and Methods] Two raters (A and B) examined 44 post-stroke patients. To assess intra-rater reliability, rater A tested subjects on three separate occasions (Days 1, 2, and 3). To assess inter-rater reliability, raters A and B independently tested participants on the same occasion (Day 3). [Results] There was no significant systematic bias between test occasions or raters. Intraclass correlation coefficient values were 0.93−0.97 for intra-rater reliability at both the comfortable speed and maximum speed, and 0.97−0.98 (comfortable speed) and 0.87−0.99 (maximum speed) for inter-rater reliability. The standard error was 1.25−1.49 (comfortable speed) and 1.62−1.77 (maximum speed) for intra-rater investigation, and 1.04−1.32 (comfortable speed) and 0.91−1.26 (maximum speed) for inter-rater investigation. At the 90% confidence level, the minimum detectable change ranged from 2.9−4.1%, and the error of an individual’s score at a given time point ranged from ±2.1−2.9%. [Conclusion] Based on this excellent reliability of the PGR in post-stroke patients, it can be recommended as a simple test of gait analysis in this population.

Activity Recognition for Persons With Stroke Using Mobile Phone Technology: Toward Improved Performance in a Home Setting

BACKGROUND

Smartphones contain sensors that measure movement-related data, making them promising tools for monitoring physical activity after a stroke. Activity recognition (AR) systems are typically trained on movement data from healthy individuals collected in a laboratory setting. However, movement patterns change after a stroke (eg, gait impairment), and activities may be performed differently at home than in a lab. Thus, it is important to validate AR for gait-impaired stroke patients in a home setting for accurate clinical predictions.

OBJECTIVE

In this study, we sought to evaluate AR performance in a home setting for individuals who had suffered a stroke, by using different sets of training activities. Specifically, we compared AR performance for persons with stroke while varying the origin of training data, based on either population (healthy persons or persons with stoke) or environment (laboratory or home setting).

METHODS

Thirty individuals with stroke and fifteen healthy subjects performed a series of mobility-related activities, either in a laboratory or at home, while wearing a smartphone. A custom-built app collected signals from the phone's accelerometer, gyroscope, and barometer sensors, and subjects self-labeled the mobility activities. We trained a random forest AR model using either healthy or stroke activity data. Primary measures of AR performance were (1) the mean recall of activities and (2) the misclassification of stationary and ambulatory activities.

RESULTS

A classifier trained on stroke activity data performed better than one trained on healthy activity data, improving average recall from 53% to 75%. The healthy-trained classifier performance declined with gait impairment severity, more often misclassifying ambulatory activities as stationary ones. The classifier trained on in-lab activities had a lower average recall for at-home activities (56%) than for in-lab activities collected on a different day (77%).

CONCLUSIONS

Stroke-based training data is needed for high quality AR among gait-impaired individuals with stroke. Additionally, AR systems for home and community monitoring would likely benefit from including at-home activities in the training data.

A Physical Activity Reference Data-Set Recorded from Older Adults Using Body-Worn Inertial Sensors and Video Technology-The ADAPT Study Data-Set

Physical activity monitoring algorithms are often developed using conditions that do not represent real-life activities, not developed using the target population, or not labelled to a high enough resolution to capture the true detail of human movement. We have designed a semi-structured supervised laboratory-based activity protocol and an unsupervised free-living activity protocol and recorded 20 older adults performing both protocols while wearing up to 12 body-worn sensors. Subjects’ movements were recorded using synchronised cameras (≥25 fps), both deployed in a laboratory environment to capture the in-lab portion of the protocol and a body-worn camera for out-of-lab activities. Video labelling of the subjects’ movements was performed by five raters using 11 different category labels. The overall level of agreement was high (percentage of agreement >90.05%, and Cohen’s Kappa, corrected kappa, Krippendorff’s alpha and Fleiss’ kappa >0.86). A total of 43.92 h of activities were recorded, including 9.52 h of in-lab and 34.41 h of out-of-lab activities. A total of 88.37% and 152.01% of planned transitions were recorded during the in-lab and out-of-lab scenarios, respectively. This study has produced the most detailed dataset to date of inertial sensor data, synchronised with high frame-rate (≥25 fps) video labelled data recorded in a free-living environment from older adults living independently. This dataset is suitable for validation of existing activity classification systems and development of new activity classification algorithms.

Evaluation of a smartphone-based assessment system in subjects with chronic ankle instability

BACKGROUND

Ankle sprain is the most common sports-related injury, and approximately 80% of patients studied suffered recurrent sprains. These repeated ankle injuries could cause chronic ankle instability, a decrease in sports performance, and a decrease in postural control ability. At the present time, smartphones have become very popular and powerful devices, and smartphone applications (apps) that have been shown to have good validity have been designed to measure human body motion. However, the app focusing on ankle function assessment and rehabilitation is still not widely used and has very limited functions. The purpose of this study is to evaluate the feasibility of smartphone-based systems in the assessment of postural control ability for patients with chronic ankle instability.

METHODS

Fifteen physically active adults (6 male, 9 female; aged = 23.4 ± 5.28 years; height = 167.13 ± 7.3 cm; weight = 62.06 ± 10.82 kg; BMI = 22.08 ± 2.57 kg/ m²) were recruited, and these participants had at least one leg that was evaluated as scoring lower than 27 points according to the Cumberland Ankle Instability Tool (CAIT). The smartphone used in the study was ASUS Zenfone 2, and an app developed using MIT App Inventor was used to record built-in accelerometer data during the assessment process. Subjects were asked to perform single leg stance for 20 s in eyes-open and eyes-closed conditions with each leg. The smartphone was fixed in an upright position on the middle of the shin, using an exercise armband, with the screen facing forward. The average of recorded acceleration data was used to represent the postural control performance, and higher values indicated more instability. Data were analyzed with a paired t-test with SPSS 17.0, and the statistical significance was set as alpha <0.05.

RESULTS

A significant difference was found between CAIT scores from the healthier leg and injured leg (healthier leg 23.07 ± 3.80 vs. injured leg 18.27 ± 3.92, p < 0.001). Significant differences were also found between the scores for the healthier leg and injured leg during both eyes-open and eyes-closed conditions (eyes-open: healthier leg 0.051 ± 0.018 vs. injured leg 0.072 ± 0.034, p = 0.027; eyes-closed: healthier leg 0.100 ± 0.031 vs. injured leg 0.123 ± 0.038, p = 0.001, unit: m/s²). Significant differences were also found between eyes-open and eyes-closed conditions during both single leg standing with healthier leg and injured leg (healthier leg: eyes-open 0.051 ± 0.018 vs. eyes-closed 0.100 ± 0.031, p < 0.001; injured leg: eyes-open 0.072 ± 0.034 vs. eyes-closed 0.123 ± 0.038, p = 0.001, unit: m/s²). The results demonstrate that the smartphone software can be used to discriminate between the different performances of the healthier leg and injured leg, and also between eyes-open and eyes-closed conditions.

CONCLUSION

The smartphone may have the potential to be a convenient, easy-to-use, and feasible tool for the assessment of postural control ability on subjects with chronic ankle instability.

A Flexible Approach for Human Activity Recognition Using Artificial Hydrocarbon Networks

Physical activity recognition based on sensors is a growing area of interest given the great advances in wearable sensors. Applications in various domains are taking advantage of the ease of obtaining data to monitor personal activities and behavior in order to deliver proactive and personalized services. Although many activity recognition systems have been developed for more than two decades, there are still open issues to be tackled with new techniques. We address in this paper one of the main challenges of human activity recognition: Flexibility. Our goal in this work is to present artificial hydrocarbon networks as a novel flexible approach in a human activity recognition system. In order to evaluate the performance of artificial hydrocarbon networks based classifier, experimentation was designed for user-independent, and also for user-dependent case scenarios. Our results demonstrate that artificial hydrocarbon networks classifier is flexible enough to be used when building a human activity recognition system with either user-dependent or user-independent approaches.