Using data from the Microsoft Kinect 2 to determine postural stability in healthy subjects: A feasibility trial

Evaluation of Arm Swing Features and Asymmetry during Gait in Parkinson's Disease Using the Azure Kinect Sensor

Arm swinging is a typical feature of human walking: Continuous and rhythmic movement of the upper limbs is important to ensure postural stability and walking efficiency. However, several factors can interfere with arm swings, making walking more risky and unstable: These include aging, neurological diseases, hemiplegia, and other comorbidities that affect motor control and coordination. Objective assessment of arm swings during walking could play a role in preventing adverse consequences, allowing appropriate treatments and rehabilitation protocols to be activated for recovery and improvement. This paper presents a system for gait analysis based on Microsoft Azure Kinect DK sensor and its body-tracking algorithm: It allows noninvasive full-body tracking, thus enabling simultaneous analysis of different aspects of walking, including arm swing characteristics. Sixteen subjects with Parkinson's disease and 13 healthy controls were recruited with the aim of evaluating differences in arm swing features and correlating them with traditional gait parameters. Preliminary results show significant differences between the two groups and a strong correlation between the parameters. The study thus highlights the ability of the proposed system to quantify arm swing features, thus offering a simple tool to provide a more comprehensive gait assessment.

Reliability of 3D Depth Motion Sensors for Capturing Upper Body Motions and Assessing the Quality of Wheelchair Transfers

Wheelchair users must use proper technique when performing sitting-pivot-transfers (SPTs) to prevent upper extremity pain and discomfort. Current methods to analyze the quality of SPTs include the TransKinect, a combination of machine learning (ML) models, and the Transfer Assessment Instrument (TAI), to automatically score the quality of a transfer using Microsoft Kinect V2. With the discontinuation of the V2, there is a necessity to determine the compatibility of other commercial sensors. The Intel RealSense D435 and the Microsoft Kinect Azure were compared against the V2 for inter- and intra-sensor reliability. A secondary analysis with the Azure was also performed to analyze its performance with the existing ML models used to predict transfer quality. The intra- and inter-sensor reliability was higher for the Azure and V2 (n = 7; ICC = 0.63 to 0.92) than the RealSense and V2 (n = 30; ICC = 0.13 to 0.7) for four key features. Additionally, the V2 and the Azure both showed high agreement with each other on the ML outcomes but not against a ground truth. Therefore, the ML models may need to be retrained ideally with the Azure, as it was found to be a more reliable and robust sensor for tracking wheelchair transfers in comparison to the V2.

Kinect-Based Assessment of Lower Limbs during Gait in Post-Stroke Hemiplegic Patients: A Narrative Review

The aim of this review was to present an overview of the state of the art in the use of the Microsoft Kinect camera to assess gait in post-stroke individuals through an analysis of the available literature. In recent years, several studies have explored the potentiality, accuracy, and effectiveness of this 3D optical sensor as an easy-to-use and non-invasive clinical measurement tool for the assessment of gait parameters in several pathologies. Focusing on stroke individuals, some of the available studies aimed to directly assess and characterize their gait patterns. In contrast, other studies focused on the validation of Kinect-based measurements with respect to a gold-standard reference (i.e., optoelectronic systems). However, the nonhomogeneous characteristics of the participants, of the measures, of the methodologies, and of the purposes of the studies make it difficult to adequately compare the results. This leads to uncertainties about the strengths and weaknesses of this technology in this pathological state. The final purpose of this narrative review was to describe and summarize the main features of the available works on gait in the post-stroke population, highlighting similarities and differences in the methodological approach and primary findings, thus facilitating comparisons of the studies as much as possible.

Multiple Participants' Discrete Activity Recognition in a Well-Controlled Environment Using Universal Software Radio Peripheral Wireless Sensing

Wireless sensing is the utmost cutting-edge way of monitoring different health-related activities and, concurrently, preserving most of the privacy of individuals. To meet future needs, multi-subject activity monitoring is in demand, whether it is for smart care centres or homes. In this paper, a smart monitoring system for different human activities is proposed based on radio-frequency sensing integrated with ensemble machine learning models. The ensemble technique can recognise a wide range of activity based on alterations in the wireless signal's Channel State Information (CSI). The proposed system operates at 3.75 GHz, and up to four subjects participated in the experimental study in order to acquire data on sixteen distinct daily living activities: sitting, standing, and walking. The proposed methodology merges subject count and performed activities, resulting in occupancy count and activity performed being recognised at the same time. To capture alterations owing to concurrent multi-subject motions, the CSI amplitudes collected from 51 subcarriers of the wireless signals were processed and merged. To distinguish multi-subject activity, a machine learning model based on an ensemble learning technique was designed and trained using the acquired CSI data. For maximum activity classes, the proposed approach attained a high average accuracy of up to 98%. The presented system has the ability to fulfil prospective health activity monitoring demands and is a viable solution towards well-being tracking.

5G-enabled contactless multi-user presence and activity detection for independent assisted living

Wireless sensing is the state-of-the-art technique for next generation health activity monitoring. Smart homes and healthcare centres have a demand for multi-subject health activity monitoring to cater for future requirements. 5G-sensing coupled with deep learning models has enabled smart health monitoring systems, which have the potential to classify multiple activities based on variations in channel state information (CSI) of wireless signals. Proposed is the first 5G-enabled system operating at 3.75 GHz for multi-subject, in-home health activity monitoring, to the best of the authors' knowledge. Classified are activities of daily life performed by up to 4 subjects, in 16 categories. The proposed system combines subject count and activities performed in different classes together, resulting in simultaneous identification of occupancy count and activities performed. The CSI amplitudes obtained from 51 subcarriers of the wireless signal are processed and combined to capture variations due to simultaneous multi-subject movements. A deep learning convolutional neural network is engineered and trained on the CSI data to differentiate multi-subject activities. The proposed system provides a high average accuracy of 91.25% for single subject movements and an overall high multi-class accuracy of 83% for 4 subjects and 16 classification categories. The proposed system can potentially fulfill the needs of future in-home health activity monitoring and is a viable alternative for monitoring public health and well being.

Automating the Clinical Assessment of Independent Wheelchair Sitting Pivot Transfer Techniques

Background

Using proper transfer technique can help to reduce forces and prevent secondary injuries. However, current assessment tools rely on the ability to subjectively identify harmful movement patterns.

Objectives

The purpose of the study was to determine the accuracy of using a low-cost markerless motion capture camera and machine learning methods to evaluate the quality of independent wheelchair sitting pivot transfers. We hypothesized that the algorithms would be able to discern proper (low risk) and improper (high risk) wheelchair transfer techniques in accordance with component items on the Transfer Assessment Instrument (TAI).

Methods

Transfer motions of 91 full-time wheelchair users were recorded and used to develop machine learning classifiers that could be used to discern proper from improper technique. The data were labeled using the TAI item scores. Eleven out of 18 TAI items were evaluated by the classifiers. Motion variables from the Kinect were inputted as the features. Random forests and k-nearest neighbors algorithms were chosen as the classifiers. Eighty percent of the data were used for model training and hyperparameter turning. The validation process was performed using 20% of the data as the test set.

Results

The area under the receiver operating characteristic curve of the test set for each item was over 0.79. After adjusting the decision threshold, the precisions of the models were over 0.87, and the model accuracies were over 71%.

Conclusion

The results show promise for the objective assessment of the transfer technique using a low cost camera and machine learning classifiers.

Final results of the NINFA project: impact of new technologies in the daily life of elderly people

BACKGROUND

The paper presents the work carried out within NINFA (iNtelligent Integrated Network For Aged people), a project for the wellbeing of the elderly people at home.

AIMS

The impact of new technologies on elderly people is evaluated with respect to the three main topics faced by NINFA.

METHODS

NINFA was structured into three main topics: (1) active user engagement from the very beginning of the planning stage: the use of specially designed questionnaires to evaluate the acceptability of new technology in general and robot caregiver specifically; (2) assessment of the well-being through non-invasive techniques: natural language processing for language change monitoring in elderly subjects; (3) automated assessment of motor and cognitive functions at home: systems to deliver tests and exergames through user interfaces compliant with elderly subjects.

RESULTS

The analysis shows that there is no a priori closure to support the technology, but it must not be invasive and must allow social interactions. The study of speech transcripts shows that a large variations in the number of words used to describe the same situation could be a sign on the onset of cognitive impairments. The specifically designed systems highlight, after the training period, significant improvements in the performances of the participants and a satisfaction with regards to the systems usability.

CONCLUSIONS

The outcomes of NINFA project highlight some important aspects of the relationship between elderly people and new technologies concerning: engagement and acceptability, assessment of the wellbeing and of the modifications of motor, cognitive and language functions.

Simplified digital balance assessment in typically developing school children

BACKGROUND

Postural balance can be considered a conjoined parameter of gross motor performance. It is acquired in early childhood and honed until adolescence, but may also be influenced by various conditions. A simplified clinical assessment of balance and posture could be helpful in monitoring motor development or therapy particularly in pediatric patients. While analogue scales are considered unprecise and lab-based force-plate posturography lacks accessibility, we propose a novel kinematic balance assessment based on markerless 3D sensor technology.

RESEARCH QUESTION

Can balance and posture be assessed by tracking kinematic data using a single 3D motion tracking camera and are the results representative of normal motor development in a healthy pediatric cohort?

METHODS

A proprietary algorithm was developed and tested that uses skeletal data from the Microsoft Kinect™ V2 3D motion capture camera to calculate and track the center of mass in real time during a set of balance tasks. The algorithm tracks the distance of the COM traveled over time to calculate a balance score (COM speed). For this study, 432 school children aged 4-18 years performed 5 balance tasks and the resulting balance scores were analyzed and correlated with demographic data.

RESULTS

Preliminary experiments demonstrated that the system was able to reliably detect differences in COM speed during different balance tasks. The method showed moderate correlation with age and sex. Athletic activity positively correlated with balance skill in the age group < 8 years, but not in older children. Body mass appeared not to be correlated with balance ability.

SIGNIFICANCE

This study demonstrates that markerless 3D motion analysis can be used for the clinical assessment of coordination and balance and could potentially be used to monitor gross motor performance at the point-of-care.

The Validity and Reliability of the Microsoft Kinect for Measuring Trunk Compensation during Reaching

Compensatory movements at the trunk are commonly utilized during reaching by persons with motor impairments due to neurological injury such as stroke. Recent low-cost motion sensors may be able to measure trunk compensation, but their validity and reliability for this application are unknown. The purpose of this study was to compare the first (K1) and second (K2) generations of the Microsoft Kinect to a video motion capture system (VMC) for measuring trunk compensation during reaching. Healthy participants (n = 5) performed reaching movements designed to simulate trunk compensation in three different directions and on two different days while being measured by all three sensors simultaneously. Kinematic variables related to reaching range of motion (ROM), planar reach distance, trunk flexion and lateral flexion, shoulder flexion and lateral flexion, and elbow flexion were calculated. Validity and reliability were analyzed using repeated-measures ANOVA, paired t-tests, Pearson’s correlations, and Bland-Altman limits of agreement. Results show that the K2 was closer in magnitude to the VMC, more valid, and more reliable for measuring trunk flexion and lateral flexion during extended reaches than the K1. Both sensors were highly valid and reliable for reaching ROM, planar reach distance, and elbow flexion for all conditions. Results for shoulder flexion and abduction were mixed. The K2 was more valid and reliable for measuring trunk compensation during reaching and therefore might be prioritized for future development applications. Future analyses should include a more heterogeneous clinical population such as persons with chronic hemiparetic stroke.

An automated, electronic assessment tool can accurately classify older adult postural stability

Current methods of balance assessment in the clinical environment are often subjective, time-consuming and lack clinical relevance for non-ambulatory older adults. The objective of this study was to develop a novel method of balance assessment that utilizes data collected using the Microsoft Kinect 2 to create a Berg Balance Scale score, which is completely determined by statistical methods rather than by human evaluators. 74 older adults, both healthy and balance impaired, were recruited for this trial. All participants completed the Berg Balance Scale (BBS) which was scored independently by trained physical therapists. Participants then completed the items of the "Modified Berg Balance Scale" in front of the Microsoft Kinect camera. Kinematic data collected during this measurement was used to train a feed-forward neural network that was used to assign a Berg Balance Scale score. The neural network model estimated the clinician-assigned BBS score to within a median of 0.93 points for the participants in our sample population (range: 0.02-5.69). Using low-cost depth sensing camera technology and a clinical protocol that takes less than 5 min to complete in both ambulatory and non-ambulatory older adults, the method outlined in this manuscript can accurately predict a participant's BBS score and thereby identify whether they are deemed a high fall risk or not. If implemented correctly, this could enable fall prevention services to be deployed in a timely fashion using low-cost, accessible technology, resulting in improved safety of older adults.

Feasibility of Home-Based Automated Assessment of Postural Instability and Lower Limb Impairments in Parkinson's Disease

A self-managed, home-based system for the automated assessment of a selected set of Parkinson’s disease motor symptoms is presented. The system makes use of an optical RGB-Depth device both to implement its gesture-based human computer interface and for the characterization and the evaluation of posture and motor tasks, which are specified according to the Unified Parkinson’s Disease Rating Scale (UPDRS). Posture, lower limb movements and postural instability are characterized by kinematic parameters of the patient movement. During an experimental campaign, the performances of patients affected by Parkinson’s disease were simultaneously scored by neurologists and analyzed by the system. The sets of parameters which best correlated with the UPDRS scores of subjects’ performances were then used to train supervised classifiers for the automated assessment of new instances of the tasks. Results on the system usability and the assessment accuracy, as compared to clinical evaluations, indicate that the system is feasible for an objective and automated assessment of Parkinson’s disease at home, and it could be the basis for the development of neuromonitoring and neurorehabilitation applications in a telemedicine framework.

Three-dimensional cameras and skeleton pose tracking for physical function assessment: A review of uses, validity, current developments and Kinect alternatives

BACKGROUND

Three-dimensional camera systems that integrate depth assessment with traditional two-dimensional images, such as the Microsoft Kinect, Intel Realsense, StereoLabs Zed and Orbecc, hold great promise as physical function assessment tools. When combined with point cloud and skeleton pose tracking software they can be used to assess many different aspects of physical function and anatomy. These assessments have received great interest over the past decade, and will likely receive further study as the integration of depth sensing and augmented reality smartphone cameras occurs more in everyday life.

RESEARCH QUESTION

The aim of this review is to discuss how these devices work, what options are available, the best methods for performing assessments and how they can be used in the future.

METHODS

Firstly, a review of the Microsoft Kinect devices and associated artificial intelligence, automated skeleton tracking algorithms is provided. This includes a narrative critique of the validity and clinical utility of these devices for assessing different aspects of physical function including spatiotemporal, kinematic and inverse dynamics data derived from gait and balance trials, and anatomical assessments performed using the depth sensor information. Methods for improving the accuracy of data are examined, including multiple-camera systems and sensor fusion with inertial monitoring units, model fitting, and marker tracking. Secondly, alternative hardware, including other structured light and time of flight methods, stereoscopic cameras and augmented reality leveraging smartphone and tablet cameras to perform measurements in three-dimensional space are summarised. Software options related to depth sensing cameras are then discussed, focussing on recent advances such as OpenPose and web-based methods such as PoseNet.

RESULTS AND SIGNIFICANCE

The clinical and non-laboratory utility of these devices holds great promise for physical function assessment, and recent developments could strengthen their ability to provide important and impactful health-related data.

An Egg Volume Measurement System Based on the Microsoft Kinect

Measuring the volume of bird eggs is a very important task for the poultry industry and ornithological research due to the high revenue generated by the industry. In this paper, we describe a prototype of a new metrological system comprising a 3D range camera, Microsoft Kinect (Version 2) and a point cloud post-processing algorithm for the estimation of the egg volume. The system calculates the egg volume directly from the egg shape parameters estimated from the least-squares method in which the point clouds of eggs captured by the Kinect are fitted to novel geometric models of an egg in a 3D space. Using the models, the shape parameters of an egg are estimated along with the egg’s position and orientation simultaneously under the least-squares criterion. Four sets of experiments were performed to verify the functionality and the performance of the system, while volumes estimated from the conventional water displacement method and the point cloud captured by a survey-grade laser scanner serve as references. The results suggest that the method is straightforward, feasible and reliable with an average egg volume estimation accuracy 93.3% when compared to the reference volumes. As a prototype, the software part of the system was implemented in a post-processing mode. However, as the proposed processing techniques is computationally efficient, the prototype can be readily transformed into a real-time egg volume system.

Eigen Posture Based Fall Risk Assessment System Using Kinect

Postural Instability (PI) is a major reason for fall in geriatric population as well as for people with diseases or disorders like Parkinson's, stroke etc. Conventional stability indicators like Berg Balance Scale (BBS) require clinical settings with skilled personnel's interventions to detect PI and finally classify the person into low, mid or high fall risk categories. Moreover these tests demand a number of functional tasks to be performed by the patient for proper assessment. In this paper a machine learning based approach is developed to determine fall risk with minimal human intervention using only Single Limb Stance exercise. The analysis is done based on the spatiotemporal dynamics of skeleton joint positions obtained from Kinect sensor. A novel posture modeling method has been applied for feature extraction along with some traditional time domain and metadata features to successfully predict the fall risk category. The proposed unobstrusive, affordable system is tested over 224 subjects and is able to achieve 75% mean accuracy on the geriatric and patient population.

Digital Analysis of Sit-to-Stand in Masters Athletes, Healthy Old People, and Young Adults Using a Depth Sensor

The aim of this study was to compare the performance between young adults (n = 15), healthy old people (n = 10), and masters athletes (n = 15) using a depth sensor and automated digital assessment framework. Participants were asked to complete a clinically validated assessment of the sit-to-stand technique (five repetitions), which was recorded using a depth sensor. A feature encoding and evaluation framework to assess balance, core, and limb performance using time- and speed-related measurements was applied to markerless motion capture data. The associations between the measurements and participant groups were examined and used to evaluate the assessment framework suitability. The proposed framework could identify phases of sit-to-stand, stability, transition style, and performance between participant groups with a high degree of accuracy. In summary, we found that a depth sensor coupled with the proposed framework could identify performance subtleties between groups.

Usability Test of Exercise Games Designed for Rehabilitation of Elderly Patients After Hip Replacement Surgery: Pilot Study

BACKGROUND

Patients who receive rehabilitation after hip replacement surgery are shown to have increased muscle strength and better functional performance. However, traditional physiotherapy is often tedious and leads to poor adherence. Exercise games, provide ways for increasing the engagement of elderly patients and increase the uptake of rehabilitation exercises.

OBJECTIVE

The objective of this study was to evaluate Fietsgame (Dutch for cycling game), which translates existing rehabilitation exercises into fun exercise games. The system connects exercise games with a patient's personal record and a therapist interface by an Internet of Things server. Thus, both the patient and physiotherapist can monitor the patient's medical status.

METHODS

This paper describes a pilot study that evaluates the usability of the Fietsgame. The study was conducted in a rehabilitation center with 9 participants, including 2 physiotherapists and 7 patients. The patients were asked to play 6 exercise games, each lasting about 5 min, under the guidance of a physiotherapist. The mean age of the patients was 74.57 years (standard deviation [SD] 8.28); all the patients were in the recovery process after hip surgery. Surveys were developed to quantitatively measure the usability factors, including presence, enjoyment, pain, exertion, and technology acceptance. Comments on advantages and suggested improvements of our game system provided by the physiotherapists and patients were summarized and their implications were discussed.

RESULTS

The results showed that after successfully playing the games, 75% to 100% of the patients experienced high levels of enjoyment in all the games except the squats game. Patients reported the highest level of exertion in squats when compared with other exercise games. Lunges resulted in the highest dropout rate (43%) due to interference with the Kinect v2 from support chairs. All the patients (100%) found the game system useful and easy to use, felt that it would be a useful tool in their further rehabilitation, and expressed that they would like to use the game in the future. The therapists indicated that the exercise games highly meet the criteria of motor rehabilitation, and they intend to continue using the game as part of their rehabilitation treatment of patients. Comments from the patients and physiotherapists suggest that real-time corrective feedback when patients perform the exercises wrongly and a more personalized user interface with options for increasing or decreasing cognitive load are needed.

CONCLUSIONS

The results suggest that Fietsgame can be used as an alternative tool to traditional motor rehabilitation for patients with hip surgery. Lunges and squats are found to be more beneficial for patients who have relatively better balance skills. A follow-up randomized controlled study will be conducted to test the effectiveness of the Fietsgame to investigate how motivating it is over a longer period of time.