A Novel Multi-Modal Teleoperation of a Humanoid Assistive Robot with Real-Time Motion Mimic

This research shows the development of a teleoperation system with an assistive robot (NAO) through a Kinect V2 sensor, a set of Meta Quest virtual reality glasses, and Nintendo Switch controllers (Joycons), with the use of the Robot Operating System (ROS) framework to implement the communication between devices. In this paper, two interchangeable operating models are proposed. An exclusive controller is used to control the robot's movement to perform assignments that require long-distance travel. Another teleoperation protocol uses the skeleton joints information readings by the Kinect sensor, the orientation of the Meta Quest, and the button press and thumbstick movements of the Joycons to control the arm joints and head of the assistive robot, and its movement in a limited area. They give image feedback to the operator in the VR glasses in a first-person perspective and retrieve the user's voice to be spoken by the assistive robot. Results are promising and can be used for educational and therapeutic purposes.

Static and dynamic validation of kinect for ergonomic postural analysis using electro-goniometers as a gold standard:A preliminary study

BACKGROUND

Evaluation of the working postures and development of new techniques are paramount in reducing the awkward postures and occurrence of musculoskeletal disorders (MSDs). The Kinect sensor, a portable and cost-effective device, appears to be a promising alternative to study work postures.

OBJECTIVE

The current study aimed to evaluate the validity of Kinect against the gold-standard instrument (electro-goniometers) for body joint angle measurements.

METHODS

A unique software application was developed to measure the critical body joint angles for postural evaluation by using the Kinect's skeletal tracking feature. The body joint angle data of ten volunteers were measured simultaneously by both Kinect and electro-goniometers. The validation analysis was conducted in both static and dynamic domains of application.

RESULTS

Minimal variation was observed between the two techniques, and the Kinect correlated well for upper-arm joint angles of 45∘, 60∘ and 90∘; lower-arm joint angles of 30∘, 45∘, 60∘, and 90∘; straight neck position, neck joint angle at maximum possible flexion; straight trunk position, trunk bend angle at full flexion. In dynamic analysis, four out of five ICC values were > 0.75 except for the upper arm. Discrepancies in the results indicated the disapproval of Kinect for only wrist measurements.

CONCLUSION

The results of the static and dynamic studies gave a sufficient basis to consider the Kinect tool as an alternative to contemporary posture-based ergonomic evaluation methods.

Depth-Based Measurement of Respiratory Volumes: A Review

Depth-based plethysmography (DPG) for the measurement of respiratory parameters is a mobile and cost-effective alternative to spirometry and body plethysmography. In addition, natural breathing can be measured without a mouthpiece, and breathing mechanics can be visualized. This paper aims at showing further improvements for DPG by analyzing recent developments regarding the individual components of a DPG measurement. Starting from the advantages and application scenarios, measurement scenarios and recording devices, selection algorithms and location of a region of interest (ROI) on the upper body, signal processing steps, models for error minimization with a reference measurement device, and final evaluation procedures are presented and discussed. It is shown that ROI selection has an impact on signal quality. Adaptive methods and dynamic referencing of body points to select the ROI can allow more accurate placement and thus lead to better signal quality. Multiple different ROIs can be used to assess breathing mechanics and distinguish patient groups. Signal acquisition can be performed quickly using arithmetic calculations and is not inferior to complex 3D reconstruction algorithms. It is shown that linear models provide a good approximation of the signal. However, further dependencies, such as personal characteristics, may lead to non-linear models in the future. Finally, it is pointed out to focus developments with respect to single-camera systems and to focus on independence from an individual calibration in the evaluation.

Reflectance Measurement Method Based on Sensor Fusion of Frame-Based Hyperspectral Imager and Time-of-Flight Depth Camera

Hyperspectral imaging and distance data have previously been used in aerial, forestry, agricultural, and medical imaging applications. Extracting meaningful information from a combination of different imaging modalities is difficult, as the image sensor fusion requires knowing the optical properties of the sensors, selecting the right optics and finding the sensors' mutual reference frame through calibration. In this research we demonstrate a method for fusing data from Fabry-Perot interferometer hyperspectral camera and a Kinect V2 time-of-flight depth sensing camera. We created an experimental application to demonstrate utilizing the depth augmented hyperspectral data to measure emission angle dependent reflectance from a multi-view inferred point cloud. We determined the intrinsic and extrinsic camera parameters through calibration, used global and local registration algorithms to combine point clouds from different viewpoints, created a dense point cloud and determined the angle dependent reflectances from it. The method could successfully combine the 3D point cloud data and hyperspectral data from different viewpoints of a reference colorchecker board. The point cloud registrations gained 0.29-0.36 fitness for inlier point correspondences and RMSE was approx. 2, which refers a quite reliable registration result. The RMSE of the measured reflectances between the front view and side views of the targets varied between 0.01 and 0.05 on average and the spectral angle between 1.5 and 3.2 degrees. The results suggest that changing emission angle has very small effect on the surface reflectance intensity and spectrum shapes, which was expected with the used colorchecker.

Sensor acquired reachable workspace in the elderly population: A cross-sectional observational study

The elderly population experiences a decline in upper extremity range of motion (ROM), impairing activities of daily living. The primary mode of quantification is by goniometer measurement. In this cross-sectional observation study, we investigate a sensor-acquired reachable workspace for assessing shoulder ROM decline in an elderly population in comparison to traditional measurements. Sixty-one healthy subjects aged ≥ 65 years were included and compared to a cohort of 39 younger subjects, aged 20 to 64. A sensor acquired reachable workspace using a Kinect motion capture camera measured the maximum reaching ability of both arms while in a seated position, measured in m2 and normalized to arm length to calculate a novel score defined as a relative surface area. This score approximates range of motion in the upper extremity. This measurement was compared to goniometer measurements, including active ROM in shoulder flexion and abduction. Total RSA shows moderate to strong correlation between goniometer in flexion and abduction in the dominant arm (R = 0.790 and R = 0.650, P < .001, respectively) and moderate correlations for the nondominant arm (R = 0.622 and R = 0.615, P < .001). Compared to the younger cohort, the elderly population demonstrated significantly reduced total RSA in the dominant arm (meanelderly = 0.774, SD = 0.09; meanyounger = 0.830, SD = 0.07, P < .001), with significant reductions in the upper lateral quadrant in both arms (dominant: meanelderly = 0.225, SD = 0.04; meanyounger = 0.241, SD = 0.01; P < .001; nondominant: meanelderly = 0.213, SD = 0.03; meanyounger = 0.228, SD = 0.01; P = .004). The test-retest reliability was strong for both dominant and nondominant total RSA (ICC > 0.762). The reachable workspace demonstrates promise as a simple and quick tool for clinicians to assess detailed and quantitative active shoulder ROM decline in the elderly population.

The Black Box of Technological Outcome Measures: An Example in Duchenne Muscular Dystrophy

Background:

Outcome measures for non-ambulant Duchenne muscular dystrophy (DMD) patients are limited, with only the Performance of the Upper Limb (PUL) approved as endpoint for clinical trials.

Objective:

We assessed four outcome measures based on devices developed for the gaming industry, aiming to overcome disadvantages of observer-dependency and motivation.

Methods:

Twenty-two non-ambulant DMD patients (range 8.6–24.1 years) and 14 healthy controls (HC; range 9.5–25.4 years) were studied at baseline and 16 patients at 12 months using Leap Motion to quantify wrist/hand active range of motion (aROM) and a Kinect sensor for reached volume with Ability Captured Through Interactive Video Evaluation (ACTIVE), Functional Workspace (FWS) summed distance to seven upper extremity body points, and trunk compensation (KinectTC). PUL 2.0 was performed in patients only. A stepwise approach assessed quality control, construct validity, reliability, concurrent validity, longitudinal change and patient perception.

Results:

Leap Motion aROM distinguished patients and HCs for supination, radial deviation and wrist flexion (range p = 0.006 to <0.001). Reliability was low and the manufacturer’s hand model did not match the sensor’s depth images. ACTIVE differed between patients and HCs (p < 0.001), correlated with PUL (rho = 0.76), and decreased over time (p = 0.030) with a standardized response mean (SRM) of –0.61. It was appraised as fun on a 10-point numeric rating scale (median 9/10). PUL decreased over time (p < 0.001) with an SRM of –1.28, and was appraised as fun (median 7/10). FWS summed distance distinguished patients and HCs (p < 0.001), but reliability in patients was insufficient. KinectTC differed between patients and HCs (p < 0.01), but correlated insufficiently with PUL (rho = –0.69).

Conclusions:

Only ACTIVE qualified as potential outcome measure in non-ambulant DMD patients, although the SRM was below the commonly used threshold of 0.8. Lack of insight in technological constraints due to intellectual property and software updates made the technology behind these outcome measures a kind of black box that could jeopardize long-term use in clinical development.

Occupational Therapy Assessment for Upper Limb Rehabilitation: A Multisensor-Based Approach

Most post-stroke patients experience varying degrees of impairment in upper limb function and fine motor skills. Occupational therapy (OT) with other rehabilitation trainings is beneficial in improving the strength and dexterity of the impaired upper limb. An accurate upper limb assessment should be conducted before prescribing upper limb OT programs. In this paper, we present a novel multisensor method for the assessment of upper limb movements that uses kinematics and physiological sensors to capture the movement of the limbs and the surface electromyogram (sEMG). These sensors are Kinect, inertial measurement unit (IMU), Xsens, and sEMG. The key assessment features of the proposed model are as follows: (1) classification of OT exercises into four classes, (2) evaluation of the quality and completion of the OT exercises, and (3) evaluation of the relationship between upper limb mobility and muscle strength in patients. According to experimental results, the overall accuracy for OT-based motion classification is 82.2%. In addition, the fusing of Kinect and Xsens data reveals that muscle strength is highly correlated with the data with a correlation coefficient (CC) of 0.88. As a result of this research, occupational therapy specialists will be able to provide early support discharge, which could alleviate the problem of the great stress that the healthcare system is experiencing today.

Quantitative Motion Measurements Based on Markerless 3D Full-Body Tracking in Children with SMA Highly Correlate with Standardized Motor Assessments

BACKGROUND

Spinal Muscular Atrophy (SMA) is the most common neurodegenerative disease in childhood. New therapeutic interventions have been developed to interrupt rapid motor deterioration. The current standard of clinical evaluation for severely weak infants is the Children's Hospital of Philadelphia Infant Test of Neuromuscular Disorders (CHOP INTEND), originally developed for SMA type 1. This test however, remains subjective and requires extensive training to be performed reliably.

OBJECTIVE

Proof of principle of the motion tracking method for capturing complex movement patterns in ten children with SMA.

METHODS

We have developed a system for tracking full-body motion in infants (KineMAT) using a commercially available, low-cost RGB-depth sensor. Ten patients with SMA (2-46 months of age; CHOP INTEND score 10-50) were recorded for 2 minutes during unperturbed spontaneous whole-body activity. Five predefined motion parameters representing 56 degrees of freedom of upper, lower extremities and trunk joints were correlated with CHOP INTEND scores using Pearson product momentum correlation (r). Test-retest analysis in two patients used descriptive statistics.

RESULTS

4/5 preselected motion parameters highly correlated with CHOP INTEND: 1. Standard deviation of joint angles (r = 0.959, test-retest range 1.3-1.9%), 2. Standard deviation of joint position (r = 0.933, test-retest range 2.9%), 3. Absolute distance of hand/foot travelled (r = 0.937, test-retest range 6-10.5%), 4. Absolute distance of hand/foot travelled against gravity (r = 0.923; test-retest range 4.8-8.5%).

CONCLUSIONS

Markerless whole-body motion capture using the KineMAT proved to objectively capture motor performance in infants and children with SMA across different severity and ages.

An evaluation of interactive projections as digital enrichment for orangutans

Digital technologies are increasingly being incorporated into the provision of enrichment for captive primates, ranging from the ad-hoc use of iPads to specifically designed hardware installed in the design of new exhibits. In this article, we present a digital enrichment system that utilizes a novel approach with interactive projections, installed in the orangutan exhibit at Melbourne Zoo. While previous research into orangutan enrichment has involved reinforcement using food rewards, this study is the first investigation of digital enrichment for orangutans that does not involve food rewards. Rather, the interactive projections were designed to provide intrinsically rewarding environmental enrichment, instead of a cheaper or more variable way of delivering food. Our observations of orangutans' use of the system supports the approach of using interactive projections, finding that it is more responsive than capacitance or resistive based touch-screen technologies to the variety of ways that orangutans "touch": with whole hands, feet, faces, and with objects. We also present the results from a preliminary study into the effect of this enrichment into orangutan behavior which found presence or use of the device did not affect time spent engaged in the majority of behaviors of the individuals studied.

Reliability and concurrent validity of TRAZER compared to three-dimensional motion capture

BACKGROUND

Efficient neural processing of visuospatial and proprioceptive input appears to be crucial for avoidance of sport injury. As such, clinically-feasible tests are needed to identify deficiencies found by advanced neuroimaging and electrophysiological tests. Three-dimensional motion capture in a laboratory setting is currently the gold standard for measurement of human movement parameters but is costly and requires extensive training. Non-immersive virtual reality systems with body motion tracking, such as TRAZER, may provide a clinically-feasible and portable means of acquiring similar variables. Test-retest reliability and concurrent validity of these systems are currently lacking.

AIM

The aim of the study was to assess the concurrent validity of the TRAZER single-camera system with 3D motion capture system and to assess the test-retest reliability of TRAZER's whole-body reactive agility metrics.

METHODS

Participants - For validity, 13 healthy individuals (24.8±3.1 years, 170.0±7.7 cm, 70.0±14.2 kg); for reliability, 18 healthy individuals (23.3±2.5 years, 168.2±11.2 cm, 78.2±17.8 kg). Design - Validity was a single-session cross-sectional study. Reliability was a 3 consecutive day test-retest study. Setting-Controlled laboratory study. Intervention - Assessments utilized randomized movements in eight directions for forty total repetitions as designated by the TRAZER system. TRAZER protocol was simultaneously tracked by Vicon Motion Capture and the TRAZER system. Reliability data were captured on three consecutive days by the TRAZER system. Main Outcome Measures - Maximum acceleration, maximum velocity, and total distance were recorded for validation. In addition to these measures, maximum deceleration, average velocity, average acceleration, average deceleration, and average reaction time were collected for reliability.

RESULTS

Overall, a lack of agreement exists between maximum outputs for TRAZER and 3D motion capture (velocity r=0.808, acceleration r=-0.090), but total distance correlation was high (r =.961). ICC values between days 1-2-3 for average measures were high (average velocity=0.847, average acceleration=0.919, and average deceleration=0.948) with the exception of average reaction time being fair (ICC=0.536). ICCs for maximum measures showed a much smaller correlation between days (velocity=0.654, acceleration=0.171, and deceleration=0.416).

CONCLUSIONS

Even though there is a lack of strong concurrent validity between measures obtained from TRAZER and 3D motion capture systems, there is strong test-retest reliability of the TRAZER system. The applicability of these findings makes TRAZER clinically relevant in scenarios requiring pre- and post-testing for return to play decisions, or monitoring of a training regimen where demonstration of validation to a gold standard measurement is not relevant.

RELEVANCE FOR PATIENTS

When test-retest capability is desired, such as in return-to-play protocols following an injury, Trazer is a reliable option.

Comparison of a Deep Learning-Based Pose Estimation System to Marker-Based and Kinect Systems in Exergaming for Balance Training

Using standard digital cameras in combination with deep learning (DL) for pose estimation is promising for the in-home and independent use of exercise games (exergames). We need to investigate to what extent such DL-based systems can provide satisfying accuracy on exergame relevant measures. Our study assesses temporal variation (i.e., variability) in body segment lengths, while using a Deep Learning image processing tool (DeepLabCut, DLC) on two-dimensional (2D) video. This variability is then compared with a gold-standard, marker-based three-dimensional Motion Capturing system (3DMoCap, Qualisys AB), and a 3D RGB-depth camera system (Kinect V2, Microsoft Inc). Simultaneous data were collected from all three systems, while participants (N = 12) played a custom balance training exergame. The pose estimation DLC-model is pre-trained on a large-scale dataset (ImageNet) and optimized with context-specific pose annotated images. Wilcoxon's signed-rank test was performed in order to assess the statistical significance of the differences in variability between systems. The results showed that the DLC method performs comparably to the Kinect and, in some segments, even to the 3DMoCap gold standard system with regard to variability. These results are promising for making exergames more accessible and easier to use, thereby increasing their availability for in-home exercise.

Effects of Kinect exergames on balance training among community older adults: A randomized controlled trial

BACKGROUND

Recent years have witnessed wide applications of exergames to balance training among the older adults. However, research concerning balance training with the use of Kinect for Xbox has remained scarce. While previous studies have shown the positive effects of exergames on improving balance and preventing falling among the older adults, there has been a paucity of empirical evidence supporting the superiority of Kinect exercise to conventional exercise over balance training among the older adults. Therefore, this study aimed to compare the feasibility, safety, and effectiveness of Kinect exercise against conventional exercise over balance training among the community older adults.

METHOD

A total of 20 participants were randomly assigned to the Kinect Exercise Group (N = 10) or the Conventional Exercise Group (N = 10) for a 5-week balance training (45 minutes a time, 2 times a week). Assessor blinding was employed to assess the participants' performance before and after the treatment, including 30-Second Chair Stand Test (30-sec CST), Timed Up and Go (TUG), Functional Reach Test (FRT), and One-Leg Stance Test (OLST) respectively with eyes open and closed. Subjective feeling of the intensity of pain and side effects were recorded throughout the investigation period. Nonparametric statistics was used for data analysis.

RESULTS

Within-group comparison between the pre-test and post-test indicated that significant differences existed in all of the 5 tests (30-sec CST, TUG, FRT, OLST with eyes open, and OLST with eyes closed) in the Kinect exercise group. To the Conventional exercise group, however, significant differences were only observed in 30-sec CST, FRT and OLST with eyes open. With regard to between-group comparison, significant differences were only found in FRT.

CONCLUSION

Such results indicated that both treatments were helpful in improving the participants' balance performance, that Kinect exercise was more effective in terms of overall balance ability, and that Kinect exercise was particularly beneficial to functional reach enhancement in comparison with traditional exercise. Kinect exercise could be a feasible, safe, and effective alternative for dynamic balance training among older adults.

Optimal Use of Titanium Dioxide Colourant to Enable Water Surfaces to Be Measured by Kinect Sensors

Recent studies have sought to use Microsoft Kinect sensors to measure water surface shape in steady flows or transient flow processes. They have typically employed a white colourant, usually titanium dioxide (TiO₂), in order to make the surface opaque and visible to the infrared-based sensors. However, the ability of Kinect Version 1 (KV1) and Kinect Version 2 (KV2) sensors to measure the deformation of ostensibly smooth reflective surfaces has never been compared, with most previous studies using a V1 sensor with no justification. Furthermore, the TiO₂ has so far been used liberally and indeterminately, with no consideration as to the type of TiO₂ to use, the optimal proportion to use or the effect it may have on the very fluid properties being measured. This paper examines the use of anatase TiO₂ with two generations of the Microsoft Kinect sensor. Assessing their performance for an ideal flat surface, it is shown that surface data obtained using the V2 sensor is substantially more reliable. Further, the minimum quantity of colourant to enable reliable surface recognition is discovered (0.01% by mass). A stability test shows that the colourant has a strong tendency to settle over time, meaning the fluid must remain well mixed, having serious implications for studies with low Reynolds number or transient processes such as dam breaks. Furthermore, the effect of TiO₂ concentration on fluid properties is examined. It is shown that previous studies using concentrations in excess of 1% may have significantly affected the viscosity and surface tension, and thus the surface behaviour being measured. It is therefore recommended that future studies employ the V2 sensor with an anatase TiO₂ concentration of 0.01%, and that the effects of TiO₂ on the fluid properties are properly quantified before any TiO₂-Kinect-derived dataset can be of practical use, for example, in validation of numerical models or in physical models of hydrodynamic processes.

Effects of a Virtual Reality Dance Training Program on Kyphosis Angle and Respiratory Parameters in Young Women With Postural Hyperkyphosis: A Randomized Controlled Clinical Trial

CONTEXT

Thoracic hyperkyphosis, one of the most common spinal deformities, may lead to undesirable pulmonary outcomes.

OBJECTIVE

To study, the efficacy of virtual reality exercise training on thoracic hyperkyphosis and respiratory parameters in young women.

DESIGN

Randomized clinical trial.

SETTING

Laboratory setting.

PARTICIPANTS AND INTERVENTION

Participants were randomly assigned to one of two 4-week exercise training groups: regular training (RT), which involved stretch and strength training, or virtual reality with RT (VRRT), which involved dance training with the Xbox 360 Kinect® game in addition to the exercises, which the RT group received.

MAIN OUTCOME MEASURES

The authors measured kyphosis angle with a Flexicurve ruler and recorded respiratory parameters (forced vital capacity [FVC] and forced expiratory volume in 1 s [FEV1]) with a spirometer in each participant at baseline and postintervention. Separate 2 × 2 repeated-measure analysis of variances were used to analyze differences between means for kyphosis angle, FEV1, and FVC. Based on the significant interactions between time and group, the paired t test was used to compare the results at baseline and postintervention, and the independent sample t test was used to compare the differences in changes between groups. Level of significance was considered at P < .05 except for paired t test that was adjusted to P < .025 for each variable in 2 groups.

RESULTS

The results showed statistically significant interactions between time and group for kyphosis angle, FEV1, and FVC. Postintervention thoracic kyphosis angle decreased and FVC increased significantly in both groups and FEV1 improved significantly in virtual reality with RT group. The improvements in thoracic kyphosis, FVC, and FEV1 in the virtual reality with RT group were significantly greater (P < .001) than in the RT group.

CONCLUSION

Dance training with the Xbox 360 Kinect game was an effective therapy in improving thoracic kyphosis angle, FVC, and FEV1 in young women with thoracic hyperkyphosis.

A RGBD-Based Interactive System for Gaming-Driven Rehabilitation of Upper Limbs

Current physiotherapy services may not be effective or suitable for certain patients due to lack of motivation, poor adherence to exercises, insufficient supervision and feedback or, in the worst case, refusal to continue with the rehabilitation plan. This paper introduces a novel approach for rehabilitation of upper limbs through KineActiv, a platform based on Microsoft Kinect v2 and developed in Unity Engine. KineActiv proposes exergames to encourage patients to perform rehabilitation exercises prescribed by a specialist, controls the patient's performance, and corrects execution errors on the fly. KineActiv comprises a web platform where the physiotherapist can review session results, monitor patient health, and adjust rehabilitation routines. We recruited 10 patients for assessing the system usability as well as the system performance. Results show that KineActiv is a usable, enjoyable and reliable system, that does not cause any negative feelings.

Kin-FOG: Automatic Simulated Freezing of Gait (FOG) Assessment System for Parkinson's Disease

Parkinson’s disease (PD) is one of the leading neurological disorders in the world with an increasing incidence rate for the elderly. Freezing of Gait (FOG) is one of the most incapacitating symptoms for PD especially in the later stages of the disease. FOG is a short absence or reduction of ability to walk for PD patients which can cause fall, reduction in patients’ quality of life, and even death. Existing FOG assessments by doctors are based on a patient’s diaries and experts’ manual video analysis which give subjective, inaccurate, and unreliable results. In the present research, an automatic FOG assessment system is designed for PD patients to provide objective information to neurologists about the FOG condition and the symptom’s characteristics. The proposed FOG assessment system uses an RGB-D sensor based on Microsoft Kinect V2 for capturing data for 5 healthy subjects who are trained to imitate the FOG phenomenon. The proposed FOG assessment system is called “Kin-FOG”. The analysis of foot joint trajectory of the motion captured by Kinect is used to find the FOG episodes. The evaluation of Kin-FOG is performed by two types of experiments, including: (1) simple walking (SW); and (2) walking with turning (WWT). Since the standing mode has features similar to a FOG episode, our Kin-FOG system proposes a method to distinguish between the FOG and standing episodes. Therefore, two general groups of experiments are conducted with standing state (WST) and without standing state (WOST). The gradient displacement of the angle between the foot and the ground is used as the feature for discriminating between FOG and standing modes. These experiments are conducted with different numbers of FOGs for getting reliable and general results. The Kin-FOG system reports the number of FOGs, their lengths, and the time slots when they occur. Experimental results demonstrate Kin-FOG has around 90% accuracy rate for FOG prediction in both experiments for different tasks (SW, WWT). The proposed Kin-FOG system can be used as a remote application at a patient’s home or a rehabilitation clinic for sending a neurologist the required FOG information. The reliability and generality of the proposed system will be evaluated for bigger data sets of actual PD subjects.

Frailty Level Monitoring and Analysis after a Pilot Six-Week Randomized Controlled Clinical Trial Using the FRED Exergame Including Biofeedback Supervision in an Elderly Day Care Centre

Background: Frailty is a status of extreme vulnerability to endogenous and exogenous stressors exposing the individual to a higher risk of negative health-related outcomes. Exercise using interactive videos, known as exergames, is being increasingly used to increase physical activity by improving health and the physical function in elderly adults. The purpose of this study is to ascertain the reduction in the degree of frailty, the degree of independence in activities of daily living, the perception of one's state of health, safety and cardiac healthiness by the exercise done using FRED over a 6-week period in elderly day care centre. Material and Methods: Frail volunteers >65 years of age, with a score of <10 points (SPPB), took part in the study. A study group and a control group of 20 participants respectively were obtained. Following randomisation, the study group (20) took part in 18 sessions in total over 6 months, and biofeedback was recorded in each session. Results: After 6 weeks, 100% of patients from the control group continued evidencing frailty risk, whereas only 5% of patients from the study group did so, with p < 0.001 statistical significance. In the case of the EQ-VAS, the control group worsened (-12.63 points) whereas the study group improved (12.05 points). The Barthel Index showed an improvement in the study group after 6 weeks, with statistically significant evidence and a value of p < 0.003906. Safety compliance with the physical activity exceeded 87% and even improved as the days went by. Discussion: Our results stand out from those obtained by other authors in that FRED is an ad hoc-designed exergame, significantly reduced the presence and severity of frailty in a sample of sedentary elders, thus potentially modifying their risk profile. It in turn improves the degree of independence in activities of daily living and the perception of one's state of health, proving to be a safe and cardiac healthy exercise. Conclusions: The study undertaken confirms the fact that the FRED game proves to be a valid technological solution for reducing frailty risk. Based on the study conducted, the exergame may be considered an effective, safe and entertaining alternative.

Can telerehabilitation games lead to functional improvement of upper extremities in individuals with Parkinson's disease?

Parkinson's disease (PD) is treated by medication, less with deep brain stimulation and physiotherapy. Different opinions on the clinical meaningfulness of the physiotherapy or recommended intensive physiotherapy were found. Our objectives were to design intensive target-based physiotherapy for upper extremities suitable for telerehabilitation services and examine the clinical meaningfulness of the exergaming at an unchanged medication plan. A telerehabilitation exergaming system using the Kinect sensor was developed; 28 patients with PD participated in the study. The system followed the participants' movements and adapted the difficulty level of the game in real time. The outcomes of the study showed that seven out of 26 participants could set up the equipment at home alone. Clinical outcomes of Box and Blocks Test (mean: 47 vs. 52, P=0.002, Cohen's d=0.40), UPDRS III (mean: 27 vs. 29, P=0.001, d=0.22), and daily activity Jebsen's test; writing a letter (mean: 24.0 vs. 20.6, P=0.003, d=0.23); and moving light objects (mean: 4.4 vs. 3.9, P=0.006, d=0.46) were statistically significant (P<0.05) and considered clinically meaningful. The Nine-Hole Peg Test showed a statistically nonsignificant improvement (mean: 28.0 vs. 26.5, P=0.089, d=0.22). The participants claimed problems with mobility but less with activities of daily living and emotional well-being (PDQ-39). The findings lead to preliminary conclusions that exergaming is feasible, but may require technical assistance, whereas clinically meaningful results could be achieved according to validated instruments and an unchanged medication plan in individuals with PD.

SIT LESS: A prototype home-based system for monitoring older adults sedentary behavior

This paper presents the overall design of a prototype home-based system aimed to reduce sedentary behavior of older adults. Quantitative performance indicators were developed to measure the sedentary behavior and daily activities of an older adult. The sedentary behavior is monitored by identifying individual positions (standing, sitting, and lying) within the field of view of a Microsoft Kinect sensor, using a custom designed algorithm. The physical activity of the older adult when outside the field of view of the Microsoft Kinect sensor is monitored by counting the number of steps using a Fitbit Charge HR watch, which the older adult is equipped with. A user interface was developed on a PC platform to interact with the older adult. The user interface is automatically operated and includes several modules. It displays the activity level, and provides feedbacks, alerts, and reminders to reduce sedentary behavior. Evaluations using a mixed methods approach that included a focus group, interviews, and observations were conducted to examine the integrated system, evaluate the users' experience with the system, and compare different types of feedbacks and alerts. The analyses indicated the feasibility of the proposed SIT LESS system along with recommendations for improving the system in future research.

Validity and reliability of POM-Checker in measuring shoulder range of motion: Protocol for a single center comparative study

INTRODUCTION

Assessments of the range of motion (ROM) in human joints have been widely used to evaluate the joint condition. Although maker based motion capture system is the most popular and practical method in the clinical field, there have been limitations such as the relatively long time required in procedure or influence of attached markers on natural movement. Recently, markerless motion capture systems have been actively developed due to their relatively lower cost and convenience for installation. The POM-Checker (Team Elysium Inc., Seoul, Rep of Korea), a markerless motion capture system, was developed with new algorithms to assess the ROM in human joints. However, the measure of the POM-Checker has not been compared with a golden-standard device in evaluating the ROM in the human joints. So we developed a protocol to evaluate the validation and reliability of the POM-Checker in measuring the shoulder ROM. This study will also provide a standard procedure to measure the shoulder ROM with the POM-Checker and 3D motion analysis system simultaneously.

METHODS/DESIGN

This protocol is for a single institution comparative study to evaluate the validity and reliability of POM-Checker. Six participants will be recruited. We will measure the angles of shoulder abduction and flexion with POM-Checker and 3D motion analysis system simultaneously. The primary outcome is the angles of shoulder abduction and flexion.

DISCUSSION

This will be the first study to analyze the validity and reliability of POM-Checker in measuring shoulder ROM. Although the sample size of this study is small, it may not influence on the results conclusively, because the measures are very precise numerical angles. Furthermore, the angles of shoulder movements will be measured with both devices simultaneously.

CONCLUSION

The results of the study will be helpful to find out the validity and reliability of a new developed ROM measure device, POM-Checker, by comparison with a golden standard system, 3D motion capture system, in measuring the shoulder ROM. It will also contribute to use of the POM-Checker in measuring the ROMs in many human joints.

Comparative analysis of respiratory motion tracking using Microsoft Kinect v2 sensor

PURPOSE

To present and evaluate a straightforward implementation of a marker-less, respiratory motion-tracking process utilizing Kinect v2 camera as a gating tool during 4DCT or during radiotherapy treatments.

METHODS

Utilizing the depth sensor on the Kinect as well as author written C# code, respiratory motion of a subject was tracked by recording depth values obtained at user selected points on the subject, with each point representing one pixel on the depth image. As a patient breathes, specific anatomical points on the chest/abdomen will move slightly within the depth image across pixels. By tracking how depth values change for a specific pixel, instead of how the anatomical point moves throughout the image, a respiratory trace can be obtained based on changing depth values of the selected pixel. Tracking these values was implemented via marker-less setup. Varian's RPM system and the Anzai belt system were used in tandem with the Kinect to compare respiratory traces obtained by each using two different subjects.

RESULTS

Analysis of the depth information from the Kinect for purposes of phase- and amplitude-based binning correlated well with the RPM and Anzai systems. Interquartile Range (IQR) values were obtained comparing times correlated with specific amplitude and phase percentages against each product. The IQR time spans indicated the Kinect would measure specific percentage values within 0.077 s for Subject 1 and 0.164 s for Subject 2 when compared to values obtained with RPM or Anzai. For 4DCT scans, these times correlate to less than 1 mm of couch movement and would create an offset of 1/2 an acquired slice.

CONCLUSION

By tracking depth values of user selected pixels within the depth image, rather than tracking specific anatomical locations, respiratory motion can be tracked and visualized utilizing the Kinect with results comparable to that of the Varian RPM and Anzai belt.

Body Weight Estimation for Dose-Finding and Health Monitoring of Lying, Standing and Walking Patients Based on RGB-D Data

This paper describes the estimation of the body weight of a person in front of an RGB-D camera. A survey of different methods for body weight estimation based on depth sensors is given. First, an estimation of people standing in front of a camera is presented. Second, an approach based on a stream of depth images is used to obtain the body weight of a person walking towards a sensor. The algorithm first extracts features from a point cloud and forwards them to an artificial neural network (ANN) to obtain an estimation of body weight. Besides the algorithm for the estimation, this paper further presents an open-access dataset based on measurements from a trauma room in a hospital as well as data from visitors of a public event. In total, the dataset contains 439 measurements. The article illustrates the efficiency of the approach with experiments with persons lying down in a hospital, standing persons, and walking persons. Applicable scenarios for the presented algorithm are body weight-related dosing of emergency patients.

Digital Analysis of Sit-to-Stand in Masters Athletes, Healthy Old People, and Young Adults Using a Depth Sensor

The aim of this study was to compare the performance between young adults (n = 15), healthy old people (n = 10), and masters athletes (n = 15) using a depth sensor and automated digital assessment framework. Participants were asked to complete a clinically validated assessment of the sit-to-stand technique (five repetitions), which was recorded using a depth sensor. A feature encoding and evaluation framework to assess balance, core, and limb performance using time- and speed-related measurements was applied to markerless motion capture data. The associations between the measurements and participant groups were examined and used to evaluate the assessment framework suitability. The proposed framework could identify phases of sit-to-stand, stability, transition style, and performance between participant groups with a high degree of accuracy. In summary, we found that a depth sensor coupled with the proposed framework could identify performance subtleties between groups.

FRED: Exergame to Prevent Dependence and Functional Deterioration Associated with Ageing. A Pilot Three-Week Randomized Controlled Clinical Trial

Introduction: Frailty syndrome and advanced age may decrease the acceptance of illness and quality of life, and worsen patients' existing health conditions, as well as leading to an increase in health care expenses. Purpose: The purpose of this study is to reduce frailty risk via the use of a FRED game which has been expressly designed and put together for the study. Materials and methods: A total of 40 frail volunteers with a score of <10 points in the short physical performance battery (SPPB) took part in a feasibility study in order to validate the FRED game. Following randomisation, the study group (20 subjects) took part in nine sessions of 20 min each over a three-week period. The control group (19 subjects) continued to lead their daily lives in the course of which they had no physical activity scheduled; Results: After three weeks and having taken part in nine physical activity sessions with the FRED game, 60% of subjects from the study group (12/20) obtained a score of ≥10 points at the end of the study, i.e., less risk of evidencing frailty. This result proved to be statistically significant (p < 0.001). The degree of compliance with and adherence to the game was confirmed by 100% attendance of the sessions. Discussion: Our findings support the hypothesis that FRED, an ad hoc designed exergame, significantly reduced the presence and severity of frailty in a sample of sedentary elders, thus potentially modifying their risk profile. Conclusions: The FRED game is a tool that shows a 99% certain improvement in the degree of frailty in frail elderly subjects. The effectiveness of the design of ad hoc games in a certain pathology or population group is therefore evidenced.

A priori patient-specific collision avoidance in radiotherapy using consumer grade depth cameras

PURPOSE

In this study, we demonstrate and evaluate a low cost, fast, and accurate avoidance framework for radiotherapy treatments. Furthermore, we provide an implementation which is patient specific and can be implemented during the normal simulation process.

METHODS

Four patients and a treatment unit were scanned with a set of consumer depth cameras to create a polygon mesh of each object. Using a fast polygon interference algorithm, the models were virtually collided to map out feasible treatment positions of the couch and gantry. The actual physical collision space was then mapped in the treatment room by moving the gantry and couch until a collision occurred with either the patient or hardware. The physical and virtual collision spaces were then compared to determine the accuracy of the system. To improve the collision predictions, a buffer geometry was added to the scanned gantry mesh and performance was assessed as a function of buffer thickness.

RESULTS

Each patient was optically scanned during simulation in less than 1 min. The average time to virtually map the collision space for 64, 800 gantry/couch states was 5.40 ± 2.88 s. The system had an average raw accuracy and negative prediction rate (NPR) across all patients of 97.3% ± 2.4% and 96.9% ± 2.2% respectively. Using a polygon buffer of 6 cm over the gantry geometry, the NPR was raised to unity for all patients, signifying the detection of all collision events. However, the average accuracy fell from 95.3% ± 3.1% to 91.5% ± 3.6% between the 3 and 6 cm buffer as more false positives were detected.

CONCLUSIONS

We successfully demonstrated a fast and low cost framework which can map an entire collision space a priori for a given patient during the time of simulation. All collisions can be avoided using polygon interference, but a polygon buffer may be required to account for geometric uncertainties of scanned objects.

Implementation of facial recognition with Microsoft Kinect v2 sensor for patient verification

PURPOSE

The aim of this study was to present a straightforward implementation of facial recognition using the Microsoft Kinect v2 sensor for patient identification in a radiotherapy setting.

MATERIALS AND METHODS

A facial recognition system was created with the Microsoft Kinect v2 using a facial mapping library distributed with the Kinect v2 SDK as a basis for the algorithm. The system extracts 31 fiducial points representing various facial landmarks which are used in both the creation of a reference data set and subsequent evaluations of real-time sensor data in the matching algorithm. To test the algorithm, a database of 39 faces was created, each with 465 vectors derived from the fiducial points, and a one-to-one matching procedure was performed to obtain sensitivity and specificity data of the facial identification system. ROC curves were plotted to display system performance and identify thresholds for match determination. In addition, system performance as a function of ambient light intensity was tested.

RESULTS

Using optimized parameters in the matching algorithm, the sensitivity of the system for 5299 trials was 96.5% and the specificity was 96.7%. The results indicate a fairly robust methodology for verifying, in real-time, a specific face through comparison from a precollected reference data set. In its current implementation, the process of data collection for each face and subsequent matching session averaged approximately 30 s, which may be too onerous to provide a realistic supplement to patient identification in a clinical setting. Despite the time commitment, the data collection process was well tolerated by all participants and most robust when consistent ambient light conditions were maintained across both the reference recording session and subsequent real-time identification sessions.

CONCLUSION

A facial recognition system can be implemented for patient identification using the Microsoft Kinect v2 sensor and the distributed SDK. In its present form, the system is accurate-if time consuming-and further iterations of the method could provide a robust, easy to implement, and cost-effective supplement to traditional patient identification methods.

Validation of an ambient measurement system (AMS) for walking speed

Walking speed is an important indicator of worsening in a variety of neurological and neuromuscular diseases, yet typically is measured only infrequently and in a clinical setting. Passive measurement of walking speed at home could provide valuable information to track the progression of many neuromuscular conditions. The purpose of this study was to validate the measurement of walking speed by a shelf-top ambient measurement system (AMS) that can be placed in a patient's home. Twenty-eight healthy adults (16 male, 12 female) were asked to walk three pre-defined routes two times each (total of 168 traversals). For each traversal, walking speed was measured simultaneously by five sources: two independent AMSs and three human timers with stopwatches. Measurements across the five sources were compared by generalised estimating equations (GEE). Correlation coefficients compared pairwise for walking speeds across the two AMSs, three human timers, and three routes all exceeded 0.86 (p < .0001), and for AMS-to-AMS exceeded 0.92 (p < .0001). Aggregated across all routes, there was no significant difference in measured walking speeds between the two AMSs (p = .596). There was a statistically significant difference between the AMSs and human timers of 8.5 cm/s (p < .0001), which is comparable to differences reported for other non-worn sensors. The tested AMS demonstrated the ability to automatically measure walking speeds comparable to manual observation and recording, which is the current standard for assessing walking speed in a clinical setting. The AMS may be used to detect changes in walking speed in community settings.

Is Technology Present in Frailty? Technology a Back-up Tool for Dealing with Frailty in the Elderly: A Systematic Review

This study analyzes the technologies used in dealing with frailty within the following areas: prevention, care, diagnosis and treatment. The aim of this paper is, on the one hand, to analyze the extent to which technology is present in terms of its relationship with frailty and what technological resources are used to treat it. Its other purpose is to define new challenges and contributions made by physiotherapy using technology. Eighty documents related to research, validation and/or the ascertaining of different types of hardware, software or both were reviewed in prominent areas. The authors used the following scales: in the area of diagnosis, Fried’s phenotype model of frailty and a model based on trials for the design of devices. The technologies developed that are based on these models accounted for 55% and 45% of cases respectively. In the area of prevention, the results proved similar regarding the use of wireless sensors with cameras (35.71%), and Kinect™ sensors (28.57%) to analyze movements and postures that indicate a risk of falling. In the area of care, results were found referring to the use of different motion, physiological and environmental wireless sensors (46,15%), i.e. so-called smart homes. In the area of treatment, the results show with a percentage of 37.5% that the Nintendo^® Wii™ console is the most used tool for treating frailty in elderly persons. Further work needs to be carried out to reduce the gap existing between technology, frail elderly persons, healthcare professionals and carers to bring together the different views about technology. This need raises the challenge of developing and implementing technology in physiotherapy via serious games that may via play and connectivity help to improve the functional capacity, general health and quality of life of frail individuals.

Microsoft Kinect-based Continuous Performance Test: An Objective Attention Deficit Hyperactivity Disorder Assessment

Background

One of the major challenges in mental medical care is finding out new instruments for an accurate and objective evaluation of the attention deficit hyperactivity disorder (ADHD). Early ADHD identification, severity assessment, and prompt treatment are essential to avoid the negative effects associated with this mental condition.

Objective

The aim of our study was to develop a novel ADHD assessment instrument based on Microsoft Kinect, which identifies ADHD cardinal symptoms in order to provide a more accurate evaluation.

Methods

A group of 30 children, aged 8-12 years (10.3 [SD 1.4]; male 70% [21/30]), who were referred to the Child and Adolescent Psychiatry Unit of the Department of Psychiatry at Fundación Jiménez Díaz Hospital (Madrid, Spain), were included in this study. Children were required to meet the Diagnostic and Statistical Manual of Mental Disorders (DSM-5) criteria of ADHD diagnosis. One of the parents or guardians of the children filled the Spanish version of the Strengths and Weaknesses of ADHD Symptoms and Normal Behavior (SWAN) rating scale used in clinical practice. Each child conducted a Kinect-based continuous performance test (CPT) in which the reaction time (RT), the commission errors, and the time required to complete the reaction (CT) were calculated. The correlations of the 3 predictors, obtained using Kinect methodology, with respect to the scores of the SWAN scale were calculated.

Results

The RT achieved a correlation of -.11, -.29, and -.37 with respect to the inattention, hyperactivity, and impulsivity factors of the SWAN scale. The correlations of the commission error with respect to these 3 factors were -.03, .01, and .24, respectively.

Conclusions

Our findings show a relation between the Microsoft Kinect-based version of the CPT and ADHD symptomatology assessed through parental report. Results point out the importance of future research on the development of objective measures for the diagnosis of ADHD among children and adolescents.

Reachable workspace and performance of upper limb (PUL) in duchenne muscular dystrophy

INTRODUCTION

The Kinect-based reachable workspace relative surface area (RSA) is compared with the performance of upper limb (PUL) assessment in Duchenne muscular dystrophy (DMD).

METHODS

29 individuals with DMD (ages: 7-23; Brooke: 1-5) underwent both Kinect-based reachable workspace RSA and PUL assessments. RSAs were also collected from 24 age-matched controls. Total and quadrant RSAs were compared with the PUL total, shoulder-, middle-, and distal-dimension scores.

RESULTS

The total reachable workspace RSA correlated well with the total PUL score (Spearman ρ = -0.602; P < 0.001), and with each of the PUL dimensional scores: shoulder (ρ = -0.624; P < 0.001), middle (ρ = -0.564; P = 0.001), and distal (ρ = -0.630; P < 0.001). With quadrant RSA, reachability in a particular quadrant was closely associated with respective PUL dimensional-level function (lateral-upper quadrant for shoulder-, lateral-upper/lower quadrants for middle-, and lateral-lower quadrant for distal-level function).

CONCLUSIONS

This study demonstrates concurrent validity of the reachable workspace outcome measure (RSA) with the DMD-specific upper extremity outcome measure (PUL).

Kinect-based rehabilitation exercises system: therapist involved approach

The Kinect-based physical rehabilitation receives increasing recognition as an approach to provide convenience for the patients who need the therapy usually from the health professions. Most of the previous studies were driven from the patients' point of view. This paper proposes a system aiming to simplify the recovery instruction from therapists, increasing patients' motivation to participate in the rehabilitation exercise. Furthermore, the architecture for developing such rehabilitation system is designed by motion capture, human action recognition and standard exercises prototype with Kinect device.

Vision-based body tracking: turning Kinect into a clinical tool

PURPOSE

Vision-based body tracking technologies, originally developed for the consumer gaming market, are being repurposed to form the core of a range of innovative healthcare applications in the clinical assessment and rehabilitation of movement ability. Vision-based body tracking has substantial potential, but there are technical limitations.

METHOD

We use our "stories from the field" to articulate the challenges and offer examples of how these can be overcome.

RESULTS

We illustrate that: (i) substantial effort is needed to determine the measures and feedback vision-based body tracking should provide, accounting for the practicalities of the technology (e.g. range) as well as new environments (e.g. home). (ii) Practical considerations are important when planning data capture so that data is analysable, whether finding ways to support a patient or ensuring everyone does the exercise in the same manner. (iii) Home is a place of opportunity for vision-based body tracking, but what we do now in the clinic (e.g. balance tests) or in the home (e.g. play games) will require modifications to achieve capturable, clinically relevant measures.

CONCLUSIONS

This article articulates how vision-based body tracking works and when it does not to continue to inspire our clinical colleagues to imagine new applications. Implications for Rehabilitation Vision-based body tracking has quickly been repurposed to form the core of innovative healthcare applications in clinical assessment and rehabilitation, but there are clinical as well as practical challenges to make such systems a reality. Substantial effort needs to go into determining what types of measures and feedback vision-based body tracking should provide. This needs to account for the practicalities of the technology (e.g. range) as well as the opportunities of new environments (e.g. the home). Practical considerations need to be accounted for when planning capture in a particular environment so that data is analysable, whether it be finding a chair substitute, ways to support a patient or ensuring everyone does the exercise in the same manner. The home is a place of opportunity with vision-based body tracking, but it would be naïve to think that we can do what we do now in the clinic (e.g. balance tests) or in the home (e.g. play games), without appropriate modifications to what constitutes a practically capturable, clinically relevant measure.