Development and Validation of a Method of Body Volume and Fat Mass Estimation Using Three-Dimensional Image Processing with a Mexican Sample

Body composition assessment using instruments such as dual X-ray densitometry (DXA) can be complex and their use is often limited to research. This cross-sectional study aimed to develop and validate a densitometric method for fat mass (FM) estimation using 3D cameras. Using two such cameras, stereographic images, and a mesh reconstruction algorithm, 3D models were obtained. The FM estimations were compared using DXA as a reference. In total, 28 adults, with a mean BMI of 24.5 (±3.7) kg/m2 and mean FM (by DXA) of 19.6 (±5.8) kg, were enrolled. The intraclass correlation coefficient (ICC) for body volume (BV) was 0.98-0.99 (95% CI, 0.97-0.99) for intra-observer and 0.98 (95% CI, 0.96-0.99) for inter-observer reliability. The coefficient of variation for kinetic BV was 0.20 and the mean difference (bias) for BV (liter) between Bod Pod and Kinect was 0.16 (95% CI, -1.2 to 1.6), while the limits of agreement (LoA) were 7.1 to -7.5 L. The mean bias for FM (kg) between DXA and Kinect was -0.29 (95% CI, -2.7 to 2.1), and the LoA was 12.1 to -12.7 kg. The adjusted R2 obtained using an FM regression model was 0.86. The measurements of this 3D camera-based system aligned with the reference measurements, showing the system's feasibility as a simpler, more economical screening tool than current systems.

Assessment of ADHD Subtypes Using Motion Tracking Recognition Based on Stroop Color-Word Tests

Attention-Deficit/Hyperactivity Disorder (ADHD) is a neurodevelopmental disorder known for its significant heterogeneity and varied symptom presentation. Describing the different subtypes as predominantly inattentive (ADHD-I), combined (ADHD-C), and hyperactive-impulsive (ADHD-H) relies primarily on clinical observations, which can be subjective. To address the need for more objective diagnostic methods, this pilot study implemented a Microsoft Kinect-based Stroop Color-Word Test (KSWCT) with the objective of investigating the potential differences in executive function and motor control between different subtypes in a group of children and adolescents with ADHD. A series of linear mixture modeling were used to encompass the performance accuracy, reaction times, and extraneous movements during the tests. Our findings suggested that age plays a critical role, and older subjects showed improvements in KSWCT performance; however, no significant divergence in activity level between the subtypes (ADHD-I and ADHD-H/C) was established. Patients with ADHD-H/C showed tendencies toward deficits in motor planning and executive control, exhibited by shorter reaction times for incorrect responses and more difficulty suppressing erroneous responses. This study provides preliminary evidence of unique executive characteristics among ADHD subtypes, advances our understanding of the heterogeneity of the disorder, and lays the foundation for the development of refined and objective diagnostic tools for ADHD.

An Internet- and Kinect-Based Multiple Sclerosis Fitness Intervention Training With Pilates Exercises: Development and Usability Study

BACKGROUND

Balance impairments are common in people with multiple sclerosis (MS), with reduced ability to maintain position and delayed responses to postural adjustments. Pilates is a popular alternative method for balance training that may reduce the rapid worsening of symptoms and the increased risk of secondary conditions (eg, depression) that are frequently associated with physical inactivity.

OBJECTIVE

In this paper, we aimed to describe the design, development, and usability testing of MS Fitness Intervention Training (MS-FIT), a Kinect-based tool implementing Pilates exercises customized for MS.

METHODS

MS-FIT has been developed using a user-centered design approach (design, prototype, user feedback, and analysis) to gain the target user's perspective. A team composed of 1 physical therapist, 2 game programmers, and 1 game designer developed the first version of MS-FIT that integrated the knowledge and experience of the team with MS literature findings related to Pilates exercises and balance interventions based on exergames. MS-FIT, developed by using the Unity 3D (Unity Technologies) game engine software with Kinect Sensor V2 for Windows, implements exercises for breathing, posture, and balance. Feedback from an Italian panel of experts in MS rehabilitation (neurologists, physiatrists, physical therapists, 1 statistician, and 1 bioengineer) and people with MS was collected to customize the tool for use in MS. The context of MS-FIT is traveling around the world to visit some of the most important cities to learn the aspects of their culture through pictures and stories. At each stay of the travel, the avatar of a Pilates teacher shows the user the exercises to be performed. Overall, 9 people with MS (n=4, 44% women; mean age 42.89, SD 11.97 years; mean disease duration 10.19, SD 9.18 years; Expanded Disability Status Scale score 3.17, SD 0.75) were involved in 3 outpatient user test sessions of 30 minutes; MS-FIT's usability was assessed through an ad hoc questionnaire (maximum value=5; higher the score, higher the usability) evaluating easiness to use, playability, enjoyment, satisfaction, and acceptance.

RESULTS

A user-centered design approach was used to develop an accessible and challenging tool for balance training. All people with MS (9/9, 100%) completed the user test sessions and answered the ad hoc questionnaire. The average score on each item ranged from 3.78 (SD 0.67) to 4.33 (SD 1.00), which indicated a high usability level. The feedback and suggestions provided by 64% (9/14) of people with MS and 36% (5/14) of therapists involved in the user test were implemented to refine the first prototype to release MS-FIT 2.0.

CONCLUSIONS

The participants reported that MS-FIT was a usable tool. It is a promising system for enhancing the motivation and engagement of people with MS in performing exercise with the aim of improving their physical status.

Kinect-based objective assessment for early frailty identification in patients with Parkinson's disease

BACKGROUND

Frailty is common in Parkinson's disease (PD) and increases vulnerability to adverse outcomes. Early detection of this syndrome aids in early intervention.

AIMS

To objectively identify frailty at an early stage during routine motor tasks in PD patients using a Kinect-based system.

METHODS

PD patients were recruited and assessed with the Fried criteria to determine their frailty status. Each participant was recorded performing the Movement Disorder Society-Sponsored Revision of the Unified Parkinson's Disease Rating Scale part III (MDS-UPDRS III) extremity tasks with a Kinect-based system. Statistically significant kinematic parameters were selected to discriminate the pre-frail from the non-frail group.

RESULTS

Of the fifty-two participants, twenty were non-frail and thirty-two were pre-frail. Decreased frequency in finger tapping (P = 0.005), hand grasping (P = 0.002), toe tapping (P = 0.002), and leg agility (P = 0.019) alongside reduced hand grasping speed (P = 0.030), lifting (P < 0.001) and falling speed (P < 0.001) in leg agility were observed in the pre-frail group. Amplitude in leg agility (P = 0.048) and amplitude decrement rate (P = 0.046) in hand grasping showed marginally significant differences between two groups. Moderate discriminative values were found in frequency and speed of the extremity tasks to identify pre-frailty with sensitivity, specificity, and area under the curve (AUC) in the range of 45.00-85.00%, 68.75-100%, and 0.701-0.836, respectively. The combination of frequency and speed in extremity tasks showed moderate to high discriminatory ability, with AUC of 0.775 (95% CI 0.637-0.913, P < 0.001) for upper limb tasks and 0.909 (95% CI 0.832-0.987, P < 0.001) for lower limb tasks. When combining these features in both upper and lower limb tasks, the AUC increased to 0.942 (95% CI 0.886-0.999, P < 0.001).

CONCLUSIONS

Our findings demonstrated the promise of utilizing Kinect-based kinematic data from MDS-UPDRS III tasks as early indicators of frailty in PD patients.

Progression events defined by home-based assessment of motor function in multiple sclerosis: protocol of a prospective study

Background

This study relates to emerging concepts of appropriate trial designs to evaluate effects of intervention on the accumulation of irreversible disability in multiple sclerosis (MS). Major starting points of our study are the known limitations of current definitions of disability progression by rater-based clinical assessment and the high relevance of gait and balance dysfunctions in MS. The study aims to explore a novel definition of disease progression using repeated instrumental assessment of relevant motor functions performed by patients in their home setting.

Methods

The study is a prospective single-center observational cohort study with the primary outcome acquired by participants themselves, a home-based assessment of motor functions based on an RGB-Depth (RGB-D) camera, a camera that provides both depth (D) and color (RGB) data. Participants are instructed to perform and record a set of simple motor tasks twice a day over a one-week period every 6 months. Assessments are complemented by a set of questionnaires. Annual research grade assessments are acquired at dedicated study visits and include clinical ratings as well as structural imaging (MRI and optical coherence tomography). In addition, clinical data from routine visits is provided semiannually by treating neurologists. The observation period is 24 months for the primary endpoint with an additional clinical assessment at 27 month to confirm progression defined by the Expanded Disability Status Scale (EDSS). Secondary analyses aim to explore the time course of changes in motor parameters and performance of the novel definition against different alternative definitions of progression in MS. The study was registered at Deutsches Register für Klinische Studien (DRKS00027042).

Discussion

The study design presented here investigates disease progression defined by marker-less home-based assessment of motor functions against 3-month confirmed disease progression (3 m-CDP) defined by the EDSS. The technical approach was chosen due to previous experience in lab-based settings. The observation time per participant of 24, respectively, 27 months is commonly conceived as the lower limit needed to study disability progression. Defining a valid digital motor outcome for disease progression in MS may help to reduce observation times in clinical trials and add confidence to the detection of progression events in MS.

Digital Motor Biomarkers of Cerebellar Ataxia Using an RGB-Depth Camera-Based Motion Analysis System

This study aimed to identify quantitative biomarkers of motor function for cerebellar ataxia by evaluating gait and postural control using an RGB-depth camera-based motion analysis system. In 28 patients with degenerative cerebellar ataxia and 33 age- and sex-matched healthy controls, motor tasks (short-distance walk, closed feet stance, and stepping in place) were selected from a previously reported protocol, and scanned using Kinect V2 and customized software. The Clinical Assessment Scale for the Assessment and Rating of Ataxia (SARA) was also evaluated. Compared with the normal control group, the cerebellar ataxia group had slower gait speed and shorter step lengths, increased step width, and mediolateral trunk sway in the walk test (all P < 0.001). Lateral sway increased in the stance test in the ataxia group (P < 0.001). When stepping in place, the ataxia group showed higher arrhythmicity of stepping and increased stance time (P < 0.001). In the correlation analyses, the ataxia group showed a positive correlation between the total SARA score and arrhythmicity of stepping in place (r = 0.587, P = 0.001). SARA total score (r = 0.561, P = 0.002) and gait subscore (ρ = 0.556, P = 0.002) correlated with mediolateral truncal sway during walking. These results suggest that the RGB-depth camera-based motion analyses on mediolateral truncal sway during walking and arrhythmicity of stepping in place are useful digital motor biomarkers for the assessment of cerebellar ataxia, and could be utilized in future clinical trials.

Feasibility of a kinect-based system in assessing physical function of the elderly for home-based care

BACKGROUND

With concerns about accurate diagnosis through telehealth, the Kinect sensor offers a reliable solution for movement analysis. However, there is a lack of practical research investigating the suitability of a Kinect-based system as a functional fitness assessment tool in homecare settings. Hence, the objective of this study was to evaluate the feasibility of using a Kinect-based system to assess physical function changes in the elderly.

METHODS

The study consisted of two phases. Phase one involved 35 young healthy adults, evaluating the reliability and validity of a Kinect-based fitness evaluation compared to traditional physical examination using the intraclass correlation coefficient (ICC). Phase two involved 665 elderly subjects, examining the correlation between the Kinect-based fitness evaluation and physical examination through Pearson's correlation coefficients. A Kinect sensor (Microsoft Xbox One Kinect V2) with customized software was employed to capture and compute the movement of joint centers. Both groups performed seven functional assessments simultaneously monitored by a physical therapist and the Kinect system. System usability and user satisfaction were assessed using the System Usability Scale (SUS) and Questionnaire for User Interface Satisfaction (QUIS), respectively.

RESULTS

Kinect-based system showed overall moderate to excellent within-day reliability (ICC = 0.633-1.0) and between-day reliability (ICC = 0.686-1.0). The overall agreement between the two devices was highly correlated (r ≧ 0.7) for all functional assessment tests in young healthy adults. The Kinect-based system also showed a high correlation with physical examination for the functional assessments (r = 0.858-0.988) except functional reach (r = 0.484) and walking speed(r = 0.493). The users' satisfaction with the system was excellent (SUS score = 84.4 ± 18.5; QUIS score = 6.5-6.7).

CONCLUSIONS

The reliability and validity of Kinect for assessing functional performance are generally favorable. Nonetheless, caution is advised when employing Kinect for tasks involving depth changes, such as functional reach and walking speed tests for their moderate validity. However, Kinect's fundamental motion detection capabilities demonstrate its potential for future applications in telerehabilitation in different healthcare settings.

Functional improvement of patients with Parkinson syndromes using a rehabilitation training software

Introduction

Individuals with Parkinsonian disorders often face limited access to specialized physiotherapy and movement training due to staff shortages and increasing disease incidence, resulting in a rapid decline in mobility and feelings of despair. Addressing these challenges requires allocating adequate resources and implementing specialized training programs to ensure comprehensive care and support. Regarding these problems, a computer software was invented that might serve as an additional home-based extension to conventional physiotherapy.

Methods

The trial took place in a rehabilitation center where every patient received equivalent treatment apart from the training program that was set up to be investigated over 3 weeks. Seventy four Patients were included and randomized between two intervention and one control group. Intervention group 1 (IG1) trained with the computer-based system two times a week while Intervention group 2 (IG2) received five training sessions a week. Using the markerless Microsoft Kinect® camera, participants controlled a digital avatar with their own body movements. UPDRS-III and Clinical measurements were performed before and after the three-week period.

Results

Patients in all groups improved in UPDRS-III pre and post intervention whereas reduction rates were higher for IG1 (-10.89%) and IG2 (-14.04%) than for CG (-7.74%). Differences between the groups were not significant (value of ps CG/IG1 0.225, CG/IG2 0.347). Growth rates for the arm abduction angle were significantly higher in IG1 (11.6%) and IG2 (9.97%) than in CG (1.87%) (value of ps CG/IG1 0.006 and CG/IG2 0.018), as was the 5-steps-distance (CG 10.86% vs. IG1 24.5% vs. UG2 26.22%, value of ps CG/IG1 0.011 and CG/IG2 0.031).

Discussion

The study shows the beneficial effects of computer-based training and substantiates the assumption of a similar impact in a home-based setting. The utilized software is feasible for such interventions and meets with the patient's approval. Group dynamics seem to have an additional supporting effect for the aspired objective of improving mobility and should be seen as an essential aspect of video games in therapy.

A deep learning-based approach for emotional analysis of sports dance

There is a phenomenon of attaching importance to technique and neglecting emotion in the training of sports dance (SP), which leads to the lack of integration between movement and emotion and seriously affects the training effect. Therefore, this article uses the Kinect 3D sensor to collect the video information of SP performers and obtains the pose estimation of SP performers by extracting the key feature points. The Arousal-Valence (AV) emotion model, based on the Fusion Neural Network model (FUSNN), is also combined with theoretical knowledge. It replaces long short term memory (LSTM) with gate recurrent unit (GRU), adds layer-normalization and layer-dropout, and reduces stack levels, and it is used to categorize SP performers' emotions. The experimental results show that the model proposed in this article can accurately detect the key points in the performance of SP performers' technical movements and has a high emotional recognition accuracy in the tasks of 4 categories and eight categories, reaching 72.3% and 47.8%, respectively. This study accurately detected the key points of SP performers in the presentation of technical movements and made a major contribution to the emotional recognition and relief of this group in the training process.

Combining 3D skeleton data and deep convolutional neural network for balance assessment during walking

Introduction: Balance impairment is an important indicator to a variety of diseases. Early detection of balance impairment enables doctors to provide timely treatments to patients, thus reduce their fall risk and prevent related disease progression. Currently, balance abilities are usually assessed by balance scales, which depend heavily on the subjective judgement of assessors. Methods: To address this issue, we specifically designed a method combining 3D skeleton data and deep convolutional neural network (DCNN) for automated balance abilities assessment during walking. A 3D skeleton dataset with three standardized balance ability levels were collected and used to establish the proposed method. To obtain better performance, different skeleton-node selections and different DCNN hyperparameters setting were compared. Leave-one-subject-out-cross-validation was used in training and validation of the networks. Results and Discussion: Results showed that the proposed deep learning method was able to achieve 93.33% accuracy, 94.44% precision and 94.46% F1 score, which outperformed four other commonly used machine learning methods and CNN-based methods. We also found that data from body trunk and lower limbs are the most important while data from upper limbs may reduce model accuracy. To further validate the performance of the proposed method, we migrated and applied a state-of-the-art posture classification method to the walking balance ability assessment task. Results showed that the proposed DCNN model improved the accuracy of walking balance ability assessment. Layer-wise Relevance Propagation (LRP) was used to interpret the output of the proposed DCNN model. Our results suggest that DCNN classifier is a fast and accurate method for balance assessment during walking.

Finding Effective Adjustment Levels for Upper Limb Exergames: Focus Group Study With Children With Physical Disabilities

BACKGROUND

We developed the Blexer system consisting of a database and a web interface for therapists that can host different types of adaptive and personally configurable virtual reality exergames based on Kinect (Microsoft Corp) motion capture to provide entertaining exercises for children with motor disabilities. It allows for parameter adjustment and the monitoring of results remotely, thereby providing a useful tool to complement traditional physical therapy sessions with home exercises.

OBJECTIVE

The aim of this study was to observe the motor benefits achieved through the use of a video exergame and the importance and implications of correctly setting the game's difficulty parameters.

METHODS

This was an observational case study of 6 children with different physical disabilities receiving physical therapy at school combined with the use of a fully configurable exergame under research that forms a part of the Blexer environment. The game integrates 4 repeatedly appearing upper limb exercises with individually adjustable difficulties (intermittent arm rising, arm forward and backward movement, rising and holding of one arm, and trunk control in all directions). The outcomes were 3 assessments of 2 efficacy measures: Box and Block Test and Jebsen Taylor Hand Function Test.

RESULTS

A total of 5 children with cerebral palsy (mean 8.4, SD 2.7 years; Gross Motor Function Classification II-2/5, 40%; III-2/5, 40%; and IV-1/5, 20%) and 1 child with obstetric brachial plexus palsy (aged 8 years; Mallet Classification III) received between 8 and 11 sessions of training (10-20 minutes per session), depending on age, motivation, and fatigue. Significant associations were observed between game parameter settings and improvements in motor function, on the one hand, and between the type of improvement and disability severity, on the other: with adjusted game parameters goal and time in the range of 70% to 100%, only less affected children improved in the Box and Block Test (+11 blocks vs -1 block), and more affected children improved more in the Jebsen Taylor Hand Function Test (+90 seconds vs +27 seconds).

CONCLUSIONS

When defining the difficulty parameters for an exergame, we suggest a classification in levels ranging from very easy to very hard. For practical use, we suggest setting the difficulty for the player to an easy or medium level rather than high-commitment goals, as this leads to a longer playtime with more fun and, therefore, seems to improve the results of the game and, consequently, mobility.

Evidence for a developmental shift in the motivation underlying helping in early childhood

We investigated children's positive emotions as an indicator of their underlying prosocial motivation. In Study 1, 2-, and 5-year-old children (N = 64) could either help an individual or watch as another person provided help. Following the helping event and using depth sensor imaging, we measured children's positive emotions through changes in postural elevation. For 2-year-olds, helping the individual and watching another person help was equally rewarding; 5-year-olds showed greater postural elevation after actively helping. In Study 2, 5-year-olds' (N = 59) positive emotions following helping were greater when an audience was watching. Together, these results suggest that 2-year-old children have an intrinsic concern that individuals be helped whereas 5-year-old children have an additional, strategic motivation to improve their reputation by helping.

[Follow control of upper limb rehabilitation training based on Kinect and NAO robot]

Gesture imitation is a common rehabilitation strategy in limb rehabilitation training. In traditional rehabilitation training, patients need to complete training actions under the guidance of rehabilitation physicians. However, due to the limited resources of the hospital, it cannot meet the training and guidance needs of all patients. In this paper, we proposed a following control method based on Kinect and NAO robot for the gesture imitation task in rehabilitation training. The method realized the joint angles mapping from Kinect coordination to NAO robot coordination through inverse kinematics algorithm. Aiming at the deflection angle estimation problem of the elbow joint, a virtual space plane was constructed and realized the accurate estimation of deflection angle. Finally, a comparative experiment for deflection angle of the elbow joint angle was conducted. The experimental results showed that the root mean square error of the angle estimation value of this method in right elbow transverse deflection and vertical deflection directions was 2.734° and 2.159°, respectively. It demonstrates that the method can follow the human movement in real time and stably using the NAO robot to show the rehabilitation training program for patients.

Ergonomic risk factors and work-related musculoskeletal disorders in clinical physiotherapy

Objectives

The purpose of this study was to objectively quantify and evaluate the ergonomic risk of clinical physiotherapy practices and evaluate physiotherapists for work-related musculoskeletal disorders and pain.

Methods

Twenty-nine physiotherapists in the rehabilitation department of a large-scale tertiary hospital were recruited in this study. The sampling period lasted for 2 weeks for each physiotherapist and interval sampling was adopted to avoid duplication of cases. Therapist posture during physiotherapy was captured, tracked and analyzed in real time using structured light sensors with an automated assessment program. The quantification of ergonomic risk was based on REBA (Rapid Entire Body Assessment) and the RPE (perceived physical exertion) scores of the therapists were recorded before and after treatment, respectively.

Results

Two hundred and twenty-four clinical physiotherapy cases were recorded, of which 49.6% were high risk and 33% were very high risk, with none of the cases presenting negligible risk. The positioning (p < 0.001) of physiotherapist had a considerable impact on ergonomic risk and pediatric physiotherapy presented a higher risk to physiotherapists than adults (p < 0.001). The RPE score of physiotherapist after performing physiotherapy was greater than before physiotherapy and was positively correlated with the REBA distribution.

Conclusion

Our study creates an automatic tool to assess the ergonomic risk of physiotherapy practices and demonstrates unacceptable ergonomic risk in common practices. The high prevalence of musculoskeletal disorders and pains recommends that rehabilitation assistance devices should be optimized and standard ergonomic courses should be included in physiotherapists' training plans.

Effectiveness of Gamification in Knee Replacement Rehabilitation: Protocol for a Randomized Controlled Trial With a Qualitative Approach

BACKGROUND

Exergames can provide encouraging exercise options. Currently, there is limited evidence regarding home-based exergaming in the postoperative phase of total knee replacement (TKR).

OBJECTIVE

This study aimed to investigate the effects of a 4-month postoperative home-based exergame intervention with an 8-month follow-up on physical function and symptoms among older persons undergoing TKR compared with home exercise using a standard protocol. In addition, a concurrent embedded design of a mixed methods study was used by including a qualitative component within a quantitative study of exergame effects.

METHODS

This was a dual-center, nonblinded, two-arm, parallel group randomized controlled trial with an embedded qualitative approach. This study aimed to recruit 100 patients who underwent their first unilateral TKR (aged 60-75 years). Participants were randomized to the exergame or standard home exercise arms. Participants followed a custom-made exergame program independently at their homes daily for 4 months. The primary outcomes at 4 months were function and pain related to the knee using the Oxford Knee Score questionnaire and mobility using the Timed Up and Go test. Other outcomes, in addition to physical function, symptoms, and disability, were game user experience, exercise adherence, physical activity, and satisfaction with the operated knee. Assessments were performed at the preoperative baseline and at 2, 4, and 12 months postoperatively. Exergame adherence was followed from game computers and using a structured diary. Self-reported standard exercise was followed for 4 months of intervention and physical activity was followed for 12 months using a structured diary. Qualitative data on patients' perspectives on rehabilitation and exergames were collected through laddering interviews at 4 and 12 months.

RESULTS

This study was funded in 2018. Data collection began in 2019 and was completed in January 2022. The COVID-19 pandemic caused an unavoidable situation in the study for recruitment, data collection, and statistical analysis. As of November 2020, a total of 52 participants had been enrolled in the study. Primary results are expected to be published by the end of 2022.

CONCLUSIONS

Our study provides new knowledge on the effects of postoperative exergame intervention among older patients with TKR. In addition, this study provides a new understanding of gamified postoperative rehabilitation, home exercise adherence, physical function, and physical activity among older adults undergoing TKR.

TRIAL REGISTRATION

ClinicalTrials.gov NCT03717727; https://clinicaltrials.gov/ct2/show/NCT03717727.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR1-10.2196/38434.

A New Architecture for Customizable Exergames: User Evaluation for Different Neuromuscular Disorders

This paper presents a modular approach to generic exergame design that combines custom physical exercises in a meaningful and motivating story. This aims to provide a tool that can be individually tailored and adapted to people with different needs, making it applicable to different diseases and states of disease. The game is based on motion capturing and integrates four example exercises that can be configured via our therapeutic web platform “Blexer-med”. To prove the feasibility for a wide range of different users, evaluation tests were performed on 14 patients with various types and degrees of neuromuscular disorders, classified into three groups based on strength and autonomy. The users were free to choose their schedule and frequency. The game scores and three surveys (before, during, and after the intervention) showed similar experiences for all groups, with the most vulnerable having the most fun and satisfaction. The players were motivated by the story and by achieving high scores. The average usage time was 2.5 times per week, 20 min per session. The pure exercise time was about half of the game time. The concept has proven feasible and forms a reasonable basis for further developments. The full 3D exercise needs further fine-tuning to enhance the fun and motivation.

Assessing Walking Stability Based on Whole-Body Movement Derived from a Depth-Sensing Camera

Stability during walking is considered a crucial aspect of assessing gait ability. The current study aimed to assess walking stability by applying principal component analysis (PCA) to decompose three-dimensional (3D) whole-body kinematic data of 104 healthy young adults (21.9 ± 3.5 years, 54 females) derived from a depth-sensing camera into a set of movement components/synergies called "principal movements" (PMs), forming together to achieve the task goal. The effect of sex as the focus area was tested on three PCA-based variables computed for each PM: the relative explained variance (rVAR) as a measure of the composition of movement structures; the largest Lyapunov exponent (LyE) as a measure of variability; and the number of zero-crossings (N) as a measure of the tightness of neuromuscular control. The results show that the sex effects appear in the specific PMs. Specifically, in PM₁, resembling the swing-phase movement, females have greater LyE (p = 0.013) and N (p = 0.017) values than males. Moreover, in PM₃, representing the mid-stance-phase movement, females have smaller rVAR (p = 0.020) but greater N (p = 0.008) values than males. These empirical findings suggest that the inherent sex differences in walking stability should be considered in assessing and training locomotion.

Use of the Kinect sensor measured three-dimensional reachable workspace to assess the upper extremity function in older adults

BACKGROUND

We explored the utility of Kinect sensor-based upper extremity reachable workspace measure in healthy adults aged over 65 years.

METHODS

Forty-three healthy older subjects (19 men and 24 women) aged over 65 years and 22 healthy young subjects (11 men and 11 women) were included. All participants were ambulatory and perform the activities of daily living independently. Three-dimensional reachable workspace data were acquired for both arms using the Kinect sensor. We evaluated hand grip strength, manual muscle shoulder strength, and the active shoulder ranges of motion of the dominant and non-dominant sides. We assessed upper limb function using the Disabilities of Arm, Shoulder, and Hand (DASH) instrument and the health-related quality of life employing the descriptive EQ-5D-5L system.

FINDINGS

The quadrant 3 relative surface area in older adults was significantly smaller than that of young adults (both dominant and non-dominant sides), while the total and quadrants 1, 2, and 4 relative surface areas did not differ between older and young adults. However, the quadrant 3 relative surface area did not correlate with the DASH or EQ5D scores. The total and quadrant 1, 2, and 4 relative surface areas of the dominant side significantly correlated with the DASH score. The quadrant 4 relative surface area of the dominant side significantly correlated with the EQ5D score.

INTERPRETATION

Kinect sensor-based, three-dimensional, reachable workspace analysis may be useful to evaluate upper limb function in older adults.

Classification of facial paralysis based on machine learning techniques

Facial paralysis (FP) is an inability to move facial muscles voluntarily, affecting daily activities. There is a need for quantitative assessment and severity level classification of FP to evaluate the condition. None of the available tools are widely accepted. A comprehensive FP evaluation system has been developed by the authors. The system extracts real-time facial animation units (FAUs) using the Kinect V2 sensor and includes both FP assessment and classification. This paper describes the development and testing of the FP classification phase. A dataset of 375 records from 13 unilateral FP patients and 1650 records from 50 control subjects was compiled. Artificial Intelligence and Machine Learning methods are used to classify seven FP categories: the normal case and three severity levels: mild, moderate, and severe for the left and right sides. For better prediction results (Accuracy = 96.8%, Sensitivity = 88.9% and Specificity = 99%), an ensemble learning classifier was developed rather than one weak classifier. The ensemble approach based on SVMs was proposed for the high-dimensional data to gather the advantages of stacking and bagging. To address the problem of an imbalanced dataset, a hybrid strategy combining three separate techniques was used. Model robustness and stability was evaluated using fivefold cross-validation. The results showed that the classifier is robust, stable and performs well for different train and test samples. The study demonstrates that FAUs acquired by the Kinect sensor can be used in classifying FP. The developed FP assessment and classification system provides a detailed quantitative report and has significant advantages over existing grading scales.

The ontogeny of children's social emotions in response to (un)fairness

Humans have a deeply rooted sense of fairness, but its emotional foundation in early ontogeny remains poorly understood. Here, we asked if and when 4- to 10-year-old children show negative social emotions, such as shame or guilt, in response to advantageous unfairness expressed through a lowered body posture (measured using a Kinect depth sensor imaging camera). We found that older, but not younger children, showed more negative emotions, i.e. a reduced upper body posture, after unintentionally disadvantaging a peer on (4,1) trials than in response to fair (1,1) outcomes between themselves and others. Younger children, in contrast, expressed more negative emotions in response to the fair (1,1) split than in response to advantageous inequity. No systematic pattern of children's emotional responses was found in a non-social context, in which children divided resources between themselves and a non-social container. Supporting individual difference analyses showed that older children in the social context expressed negative emotions in response to advantageous inequity without directly acting on this negative emotional response by rejecting an advantageously unfair offer proposed by an experimenter at the end of the study. These findings shed new light on the emotional foundation of the human sense of fairness and suggest that children's negative emotional response to advantageous unfairness developmentally precedes their rejection of advantageously unfair resource distributions.

The ontogeny of children's social emotions in response to (un)fairness

Humans have a deeply rooted sense of fairness, but its emotional foundation in early ontogeny remains poorly understood. Here, we asked if and when 4- to 10-year-old children show negative social emotions, such as shame or guilt, in response to advantageous unfairness expressed through a lowered body posture (measured using a Kinect depth sensor imaging camera). We found that older, but not younger children, showed more negative emotions, i.e. a reduced upper body posture, after unintentionally disadvantaging a peer on (4,1) trials than in response to fair (1,1) outcomes between themselves and others. Younger children, in contrast, expressed more negative emotions in response to the fair (1,1) split than in response to advantageous inequity. No systematic pattern of children's emotional responses was found in a non-social context, in which children divided resources between themselves and a non-social container. Supporting individual difference analyses showed that older children in the social context expressed negative emotions in response to advantageous inequity without directly acting on this negative emotional response by rejecting an advantageously unfair offer proposed by an experimenter at the end of the study. These findings shed new light on the emotional foundation of the human sense of fairness and suggest that children's negative emotional response to advantageous unfairness developmentally precedes their rejection of advantageously unfair resource distributions.

Brain PET motion correction using 3D face-shape model: the first clinical study

OBJECTIVE

Head motions during brain PET scan cause degradation of brain images, but head fixation or external-maker attachment become burdensome on patients. Therefore, we have developed a motion correction method that uses a 3D face-shape model generated by a range-sensing camera (Kinect) and by CT images. We have successfully corrected the PET images of a moving mannequin-head phantom containing radioactivity. Here, we conducted a volunteer study to verify the effectiveness of our method for clinical data.

METHODS

Eight healthy men volunteers aged 22-45 years underwent a 10-min head-fixed PET scan as a standard of truth in this study, which was started 45 min after ¹⁸F-fluorodeoxyglucose (285 ± 23 MBq) injection, and followed by a 15-min head-moving PET scan with the developed Kinect based motion-tracking system. First, selecting a motion-less period of the head-moving PET scan provided a reference PET image. Second, CT images separately obtained on the same day were registered to the reference PET image, and create a 3D face-shape model, then, to which Kinect-based 3D face-shape model matched. This matching parameter was used for spatial calibration between the Kinect and the PET system. This calibration parameter and the motion-tracking of the 3D face shape by Kinect comprised our motion correction method. The head-moving PET with motion correction was compared with the head-fixed PET images visually and by standard uptake value ratios (SUVRs) in the seven volume-of-interest regions. To confirm the spatial calibration accuracy, a test-retest experiment was performed by repeating the head-moving PET with motion correction twice where the volunteer's pose and the sensor's position were different.

RESULTS

No difference was identified visually and statistically in SUVRs between the head-moving PET images with motion correction and the head-fixed PET images. One of the small nuclei, the inferior colliculus, was identified in the head-fixed PET images and in the head-moving PET images with motion correction, but not in those without motion correction. In the test-retest experiment, the SUVRs were well correlated (determinant coefficient, r² = 0.995).

CONCLUSION

Our motion correction method provided good accuracy for the volunteer data which suggested it is useable in clinical settings.

Effectiveness and Usability of a Novel Kinect-Based Tailored Interactive Fall Intervention System for Fall Prevention in Older People: A Preliminary Study

Falls are prevalent among older people and can lead to serious health problems. We newly developed a novel Kinect-based tailored interactive fall intervention system, which seamlessly integrates multifactorial fall risk assessment and tailored intervention programs to prevent falls in older people. This preliminary study aimed to examine the effectiveness and usability of this developed system for fall prevention in older people. Thirty community-dwelling older women participated in this experiment; they were allocated to an intervention group (IG) or a control group (CG) for a quasi-randomized trial (15 people each). Participants in IG followed an 8-week tailored intervention (40 min/session × 2 sessions/week × 8 weeks) using the Kinect-based interactive fall intervention system, while participants in CG maintained their habitual activities. Various outcome measures were evaluated at baseline (Week 0), interim (Week 4), and post-intervention (Week 8). Experimental results showed that IG led to significant improvements in TUG-Timed Up and Go (p = 0.010), BBS-Berg Balance Scale (p = 0.011), and Montreal Cognitive Assessment-MoCA (p = 0.022) between baseline and post-intervention. In comparison to the baseline, TUG and BBS were even significantly improved at interim (p = 0.004 and 0.047, respectively). There were no significant changes in static balance-related performance outcomes and the Short Falls Efficacy Scale-SFES after the intervention. Whereas in CG, most performance measures did not show significant changes during the 8-week period, TUG completion time became significantly longer at post-intervention in comparison to interim (p = 0.028) and fear of falling was also significantly higher at post-intervention than baseline (p = 0.021). These findings suggest that the Kinect-based 8-week tailored interactive fall interventions effectively improved older people's physical and cognitive abilities. Regarding the usability of the developed system, the average System Usability Scale (SUS) score was 83.5 out of 100, indicating excellent system usability. The overall mean Computer Literacy Scale (CLS) score was 2.5 out of 26, showing that older participants in this study had very limited experience with computers. No significant correlation between SUS and CLS scores demonstrated that newly developed Kinect-based tailored interactive fall intervention system was easy to use for older people, regardless of their computer experience. This novel system should help health professionals and older people proactively manage the risk of falls.

Ground reaction force and joint moment estimation during gait using an Azure Kinect-driven musculoskeletal modeling approach

BACKGROUND

Gait analysis is burdened by time and equipment costs, interpretation, and accessibility of three-dimensional motion analysis systems. Evidence suggests growing adoption of gait testing in the shift toward evidence-based medicine. Further developments addressing these barriers will aid its efficacy in clinical practice. Previous research aiming to develop gait analysis systems for kinetics estimation using the Kinect V2 have provided promising results yet modified approaches using the latest hardware may further aid kinetics estimation accuracy RESEARCH QUESTION: Can a single Azure Kinect sensor combined with a musculoskeletal modeling approach provide kinetics estimations during gait similar to those obtained from marker-based systems with embedded force platforms?

METHODS

Ten subjects were recruited to perform three walking trials at their normal speed. Trials were recorded using an eight-camera optoelectronic system with two embedded force plates and a single Azure Kinect sensor. Marker and depth data were both used to drive a musculoskeletal model using the AnyBody Modeling System. Predicted kinetics from the Azure Kinect-driven model, including ground reaction force (GRF) and joint moments, were compared to measured values using root meansquared error (RMSE), normalized RMSE, Pearson correlation, concordance correlation, and statistical parametric mapping RESULTS: High to very high correlations were observed for anteroposterior GRF (ρ = 0.889), vertical GRF (ρ = 0.940), and sagittal hip (ρ = 0.805) and ankle (ρ = 0.876) moments. RMSEs were 1.2 ± 2.2 (%BW), 3.2 ± 5.7 (%BW), 0.7 ± 0.1.3 (%BWH), and 0.6 ± 1.0 (%BWH) SIGNIFICANCE: The proposed approach using the Azure Kinect provided higher accuracy compared to previous studies using the Kinect V2 potentially due to improved foot tracking by the Azure Kinect. Future studies should seek to optimize ground contact parameters and focus on regions of error between predicted and measured kinetics highlighted currently for further improvements in kinetic estimations.

Automatic Personality Assessment through Movement Analysis

Obtaining accurate and objective assessments of an individual's personality is vital in many areas including education, medicine, sports and management. Currently, most personality assessments are conducted using scales and questionnaires. Unfortunately, it has been observed that both scales and questionnaires present various drawbacks. Their limitations include the lack of veracity in the answers, limitations in the number of times they can be administered, or cultural biases. To solve these problems, several articles have been published in recent years proposing the use of movements that participants make during their evaluation as personality predictors. In this work, a multiple linear regression model was developed to assess the examinee's personality based on their movements. Movements were captured with the low-cost Microsoft Kinect camera, which facilitates its acceptance and implementation. To evaluate the performance of the proposed system, a pilot study was conducted aimed at assessing the personality traits defined by the Big-Five Personality Model. It was observed that the traits that best fit the model are Extroversion and Conscientiousness. In addition, several patterns that characterize the five personality traits were identified. These results show that it is feasible to assess an individual's personality through his or her movements and open up pathways for several research.

Comparison of Motion Analysis Systems in Tracking Upper Body Movement of Myoelectric Bypass Prosthesis Users

Current literature lacks a comparative analysis of different motion capture systems for tracking upper limb (UL) movement as individuals perform standard tasks. To better understand the performance of various motion capture systems in quantifying UL movement in the prosthesis user population, this study compares joint angles derived from three systems that vary in cost and motion capture mechanisms: a marker-based system (Vicon), an inertial measurement unit system (Xsens), and a markerless system (Kinect). Ten healthy participants (5F/5M; 29.6 ± 7.1 years) were trained with a TouchBionic i-Limb Ultra myoelectric terminal device mounted on a bypass prosthetic device. Participants were simultaneously recorded with all systems as they performed standardized tasks. Root mean square error and bias values for degrees of freedom in the right elbow, shoulder, neck, and torso were calculated. The IMU system yielded more accurate kinematics for shoulder, neck, and torso angles while the markerless system performed better for the elbow angles. By evaluating the ability of each system to capture kinematic changes of simulated upper limb prosthesis users during a variety of standardized tasks, this study provides insight into the advantages and limitations of using different motion capture technologies for upper limb functional assessment.

A system for real-time multivariate feature combination of endoscopic mitral valve simulator training data

Purpose

For an in-depth analysis of the learning benefits that a stereoscopic view presents during endoscopic training, surgeons required a custom surgical evaluation system enabling simulator independent evaluation of endoscopic skills. Automated surgical skill assessment is in dire need since supervised training sessions and video analysis of recorded endoscope data are very time-consuming. This paper presents a first step towards a multimodal training evaluation system, which is not restricted to certain training setups and fixed evaluation metrics.

Methods

With our system we performed data fusion of motion and muscle-action measurements during multiple endoscopic exercises. The exercises were performed by medical experts with different surgical skill levels, using either two or three-dimensional endoscopic imaging. Based on the multi-modal measurements, training features were calculated and their significance assessed by distance and variance analysis. Finally, the features were used automatic classification of the used endoscope modes.

Results

During the study, 324 datasets from 12 participating volunteers were recorded, consisting of spatial information from the participants’ joint and right forearm electromyographic information. Feature significance analysis showed distinctive significance differences, with amplitude-related muscle information and velocity information from hand and wrist being among the most significant ones. The analyzed and generated classification models exceeded a correct prediction rate of used endoscope type accuracy rate of 90%.

Conclusion

The results support the validity of our setup and feature calculation, while their analysis shows significant distinctions and can be used to identify the used endoscopic view mode, something not apparent when analyzing time tables of each exercise attempt. The presented work is therefore a first step toward future developments, with which multivariate feature vectors can be classified automatically in real-time to evaluate endoscopic training and track learning progress.

Supplementary Information

The online version contains supplementary material available at 10.1007/s11548-022-02588-1.

Automatic Decision-Making Style Recognition Method Using Kinect Technology

In recent years, somatosensory interaction technology, represented by Microsoft's Kinect hardware platform, has been widely used in various fields, such as entertainment, education, and medicine. Kinect technology can easily capture and record behavioral data, which provides new opportunities for behavioral and psychological correlation analysis research. In this paper, an automatic decision-style recognition method is proposed. Experiments involving 240 subjects were conducted to obtain face data and individual decision-making style score. The face data was obtained using the Kinect camera, and the decision-style score were obtained via a questionnaire. To realize automatic recognition of an individual decision-making style, machine learning was employed to establish the mapping relationship between the face data and a scaled evaluation of the decision-making style score. This study adopts a variety of classical machine learning algorithms, including Linear regression, Support vector machine regression, Ridge regression, and Bayesian ridge regression. The experimental results show that the linear regression model returns the best results. The correlation coefficient between the linear regression model evaluation results and the scale evaluation results was 0.6, which represents a medium and higher correlation. The results verify the feasibility of automatic decision-making style recognition method based on facial analysis.

Feasibility of a virtual reality-based exercise intervention and low-cost motion tracking method for estimation of motor proficiency in youth with autism spectrum disorder

Background

Motor impairment is widely acknowledged as a core feature in children with autism spectrum disorder (ASD), which can affect adaptive behavior and increase severity of symptoms. Low-cost motion capture and virtual reality (VR) game technologies hold a great deal of promise for providing personalized approaches to motor intervention in ASD. The present study explored the feasibility, acceptability and potential efficacy of a custom-designed VR game-based intervention (GaitWayXR™) for improving gross motor skills in youth with ASD.

Methods

Ten children and adolescents (10–17 years) completed six, 20-min VR-based motor training sessions over 2 weeks while whole-body movement was tracked with a low-cost motion capture system. We developed a methodology for using motion tracking data to quantify whole-body movement in terms of efficiency, synchrony and symmetry. We then studied the relationships of the above quantities with standardized measures of motor skill and cognitive flexibility.

Results

Our results supported our presumption that the VR intervention is safe, with no adverse events and very few minor to moderate side-effects, while a large proportion of parents said they would use the VR game at home, the most prohibitive reasons for adopting the system for home therapy were cost and space. Although there was little evidence of any benefits of the GaitWayXR™ intervention in improving gross motor skills, we showed several positive correlations between the standardized measures of gross motor skills in ASD and our measures of efficiency, symmetry and synchrony from low-cost motion capture.

Conclusions

These findings, though preliminary and limited by small sample size, suggest that low-cost motion capture of children with ASD is feasible with movement exercises in a VR-based game environment. Based on these preliminary findings, we recommend conducting larger-scale studies with methods for improving adherence to VR gaming interventions over longer periods.

Baseline drift vector of multiple points on body surface using a near-infrared camera

The purpose of this study was to extract the three-dimensional (3D) vector of the baseline drift baseline drift vector (BDV) of the specific points on the body surface and to demonstrate the importance of the 3D tracking of the body surface. Our system consisted of a near-infrared camera (NIC: Kinect V2) and software that recognized and tracked blue stickers as markers. We acquired 3D coordinates of 30 markers stuck on the body surface for 30 min for eight healthy volunteers and developed a simple technique to extract the BDV. The BDV on the sternum, rib, and abdomen was extracted from the measured data. BDV per min. was analyzed to estimate the frequency to exceed a given tolerance. Also, the correlation among BDVs for multiple body sites was analyzed. The longitudinal baseline drift was observed in the BDV of healthy volunteers. Among the eight volunteers, the maximum probability that the BDV per min. exceeded the tolerance of 1 mm and 2 mm was 30% and 15%, respectively. The correlation among BDVs of multiple body sites suggested a potential feasibility to distinguish the translational movement of the whole area and the respiratory movement. In conclusion, we constructed the 3D tracking system of multiple points on the body surface using a noninvasive NIC at a low cost and established the method to extract the BDV. The existence of the longitudinal baseline drift showed the importance of the 3D tracking in the body surface.

The Reliability of the Microsoft Kinect and Ambulatory Sensor-Based Motion Tracking Devices to Measure Shoulder Range-of-Motion: A Systematic Review and Meta-Analysis

Advancements in motion sensing technology can potentially allow clinicians to make more accurate range-of-motion (ROM) measurements and informed decisions regarding patient management. The aim of this study was to systematically review and appraise the literature on the reliability of the Kinect, inertial sensors, smartphone applications and digital inclinometers/goniometers to measure shoulder ROM. Eleven databases were screened (MEDLINE, EMBASE, EMCARE, CINAHL, SPORTSDiscus, Compendex, IEEE Xplore, Web of Science, Proquest Science and Technology, Scopus, and PubMed). The methodological quality of the studies was assessed using the consensus-based standards for the selection of health Measurement Instruments (COSMIN) checklist. Reliability assessment used intra-class correlation coefficients (ICCs) and the criteria from Swinkels et al. (2005). Thirty-two studies were included. A total of 24 studies scored "adequate" and 2 scored "very good" for the reliability standards. Only one study scored "very good" and just over half of the studies (18/32) scored "adequate" for the measurement error standards. Good intra-rater reliability (ICC > 0.85) and inter-rater reliability (ICC > 0.80) was demonstrated with the Kinect, smartphone applications and digital inclinometers. Overall, the Kinect and ambulatory sensor-based human motion tracking devices demonstrate moderate-good levels of intra- and inter-rater reliability to measure shoulder ROM. Future reliability studies should focus on improving study design with larger sample sizes and recommended time intervals between repeated measurements.

Automated and accurate assessment for postural abnormalities in patients with Parkinson's disease based on Kinect and machine learning

Background

Automated and accurate assessment for postural abnormalities is necessary to monitor the clinical progress of Parkinson’s disease (PD). The combination of depth camera and machine learning makes this purpose possible.

Methods

Kinect was used to collect the postural images from 70 PD patients. The collected images were processed to extract three-dimensional body joints, which were then converted to two-dimensional body joints to obtain eight quantified coronal and sagittal features (F1-F8) of the trunk. The decision tree classifier was carried out over a data set established by the collected features and the corresponding doctors’ MDS-UPDRS-III 3.13 (the 13th item of the third part of Movement Disorder Society-Sponsored Revision of the Unified Parkinson’s Disease Rating Scale) scores. An objective function was implanted to further improve the human–machine consistency.

Results

The automated grading of postural abnormalities for PD patients was realized with only six selected features. The intraclass correlation coefficient (ICC) between the machine’s and doctors’ score was 0.940 (95%CI, 0.905–0.962), meaning the machine was highly consistent with the doctors’ judgement. Besides, the decision tree classifier performed outstandingly, reaching 90.0% of accuracy, 95.7% of specificity and 89.1% of sensitivity in rating postural severity.

Conclusions

We developed an intelligent evaluation system to provide accurate and automated assessment of trunk postural abnormalities in PD patients. This study demonstrates the practicability of our proposed method in the clinical scenario to help making the medical decision about PD.

Upper Limb Movement Measurement Systems for Cerebral Palsy: A Systematic Literature Review

Quantifying the quality of upper limb movements is fundamental to the therapeutic process of patients with cerebral palsy (CP). Several clinical methods are currently available to assess the upper limb range of motion (ROM) in children with CP. This paper focuses on identifying and describing available techniques for the quantitative assessment of the upper limb active range of motion (AROM) and kinematics in children with CP. Following the screening and exclusion of articles that did not meet the selection criteria, we analyzed 14 studies involving objective upper extremity assessments of the AROM and kinematics using optoelectronic devices, wearable sensors, and low-cost Kinect sensors in children with CP aged 4–18 years. An increase in the motor function of the upper extremity and an improvement in most of the daily tasks reviewed were reported. In the population of this study, the potential of wearable sensors and the Kinect sensor natural user interface as complementary devices for the quantitative evaluation of the upper extremity was evident. The Kinect sensor is a clinical assessment tool with a unique markerless motion capture system. Few authors had described the kinematic models and algorithms used to estimate their kinematic analysis in detail. However, the kinematic models in these studies varied from 4 to 10 segments. In addition, few authors had followed the joint assessment recommendations proposed by the International Society of Biomechanics (ISB). This review showed that three-dimensional analysis systems were used primarily for monitoring and evaluating spatiotemporal variables and kinematic parameters of upper limb movements. The results indicated that optoelectronic devices were the most commonly used systems. The joint assessment recommendations proposed by the ISB should be used because they are approved standards for human kinematic assessments. This review was registered in the PROSPERO database (CRD42021257211).

Artificial neural network can improve the accuracy of a markerless skeletal model in L5/S1 position estimation during symmetric lifting

This study investigated whether using an artificial neural network (ANN) method for L5/S1 position estimation based on the Kinect markerless skeletal model can produce more accurate data than measurements using the original Kinect skeletal model during symmetric lifting tasks. Twenty participants performed three symmetric lifting tasks twice at three vertical lifting height paths. Their postural data were simultaneously collected by a Kinect and a reference motion tracking system (MTS). The Kinect-based data are used as the model inputs, while its outputs are based on MTS. Three-layer ANN models to predict the L5/S1 position over the entire lifting duration were trained by identifying the relationship between the seven inputs (the participant's height and weight and the Kinect-based trunk angle, left knee angle, and left hip joint coordinates on the X-axis, Y-axis, and Z-axis) and three outputs (the reference L5/S1 position on the X-axis, Y-axis, and Z-axis). As a measure of error, the distances between the reference anatomical L5/S1 position and the predicted positions (by the ANN-Kinect system and the Kinect system) were calculated and compared. The results showed that introducing the ANN method can significantly (p < 0.0001) reduce the L5/S1 position estimation error (5.12 ± 1.83 cm) in comparison with directly using the original data output from the skeletal model driven by Kinect data (20.54 ± 3.24 cm). This method provides an alternative for L5/S1 position estimation while retaining the advantages of using Kinect such as portability, easy of use, and being equipped with the function of automatic skeletal identification.

Using Kinect body joint detection system to predict energy expenditures during physical activities

The purpose of this study was to explore the potential of Kinect body joint detection to facilitate the calculation of energy expenditure during exergame exercises. Two Kinect-based biomechanical models - mechanical energy (KineticE) and work (WorkE) were employed to estimate the energy expenditure during four Wii™ exergame session. Consequently, two stepwise regression models were developed from nineteen participants' data and then validated by five holdout participants. The data collected using an accelerometer (r = 0.835, p < 0.001) had the highest correlation as compared to that of the WorkE (r = 0.805, p < 0.001) and KineticE (r = 0.466, p < 0.001) correlations with the reference indirect calorimetry using Quark activity energy expenditure (QuarkAEE). The regression results show that KineticE and the weight of the participant were significant factors for mechanical energy prediction (AEEKinetic). However, according to the work prediction equation (AEEWork), only WorkE was significant. The new energy prediction models showed significant agreement with the standard QuarkAEE (AEEKinect, r = 0.641, p = 0.02; AEEWork, r = 0.793, p < 0.001), and they were comparable to accelerometer predictions (r = 0.682, p = 0.001). The findings indicate that Kinect can be a potentially viable alternative to measure energy expenditures. The models can be applied with higher accuracy, especially when the activity demands high body movements.

Assessment of upper limb abnormalities using the Kinect: Reliability, validity and detection accuracy in people living with acquired brain injury

Upper limb kinematic abnormalities are prevalent in people with acquired brain injury (ABI). We examined if the Microsoft Kinect for Xbox One (Kinect) reliably (test-retest) and validly (concurrent) quantifies upper limb kinematics, and accurately classifies abnormalities (sensitivity/specificity), in an ABI cohort when compared to three-dimensional motion analysis (3DMA) and a subjective rating scale. We compared 42 adults with ABI to 36 healthy control (HC) participants. Walking trials were recorded by 3DMA and Kinect at self-selected (SSWS) and fast (FWS) walking speeds. When classifying abnormalities for 3DMA and Kinect, a 95% reference range (based on HC data) was calculated using the Kinematic Deviation Score worst axis (KDSw); values outside of this range were classified abnormal. Scores ≥ 2 in the subjective rating scale, based on International Classification of Functioning, Disability and Health Framework's Qualifiers Scale, were considered abnormal. Test-retest reliability and concurrent validity were determined using intra-class correlation coefficient (Absolute ICC_2,1) and Pearson's or Spearman's correlation respectively. Fisher's Exact Test was conducted to determine sensitivity and specificity between each combination of the two methods. Strong test-retest reliability was observed for 3DMA (median(IQR) ICC:0.86(0.85-0.90)). Kinect showed overall strong SSWS test-retest reliability (ICC:0.87(0.84-0.91)) and moderate FWS test-retest reliability (ICC:0.61(0.56-0.65)). Concurrent validity between 3DMA and Kinect was overall moderate. Sensitivity and specificity between 3DMA, Kinect and subjective scores were overall modest. Our results suggest caution should be used if implementing Kinect as its validity is modest against criterion-reference 3DMA; however, given its reliability and similar sensitivity/specificity to 3DMA further responsiveness research is warranted.

Placement Recommendations for Single Kinect-Based Motion Capture System in Unilateral Dynamic Motion Analysis

Low-cost, portable, and easy-to-use Kinect-based systems achieved great popularity in out-of-the-lab motion analysis. The placement of a Kinect sensor significantly influences the accuracy in measuring kinematic parameters for dynamics tasks. We conducted an experiment to investigate the impact of sensor placement on the accuracy of upper limb kinematics during a typical upper limb functional task, the drinking task. Using a 3D motion capture system as the golden standard, we tested twenty-one Kinect positions with three different distances and seven orientations. Upper limb joint angles, including shoulder flexion/extension, shoulder adduction/abduction, shoulder internal/external rotation, and elbow flexion/extension angles, are calculated via our developed Kinect kinematic model and the UWA kinematic model for both the Kinect-based system and the 3D motion capture system. We extracted the angles at the point of the target achieved (PTA). The mean-absolute-error (MEA) with the standard represents the Kinect-based system's performance. We conducted a two-way repeated measure ANOVA to explore the impacts of distance and orientation on the MEAs for all upper limb angles. There is a significant main effect for orientation. The main effects for distance and the interaction effects do not reach statistical significance. The post hoc test using LSD test for orientation shows that the effect of orientation is joint-dependent and plane-dependent. For a complex task (e.g., drinking), which involves body occlusions, placing a Kinect sensor right in front of a subject is not a good choice. We suggest that place a Kinect sensor at the contralateral side of a subject with the orientation around 30∘ to 45∘ for upper limb functional tasks. For all kinds of dynamic tasks, we put forward the following recommendations for the placement of a Kinect sensor. First, set an optimal sensor position for capture, making sure that all investigated joints are visible during the whole task. Second, sensor placement should avoid body occlusion at the maximum extension. Third, if an optimal location cannot be achieved in an out-of-the-lab environment, researchers could put the Kinect sensor at an optimal orientation by trading off the factor of distance. Last, for those need to assess functions of both limbs, the users can relocate the sensor and re-evaluate the functions of the other side once they finish evaluating functions of one side of a subject.

Machine Learning Approach for Fatigue Estimation in Sit-to-Stand Exercise

Physical exercise (PE) has become an essential tool for different rehabilitation programs. High-intensity exercises (HIEs) have been demonstrated to provide better results in general health conditions, compared with low and moderate-intensity exercises. In this context, monitoring of a patients' condition is essential to avoid extreme fatigue conditions, which may cause physical and physiological complications. Different methods have been proposed for fatigue estimation, such as: monitoring the subject's physiological parameters and subjective scales. However, there is still a need for practical procedures that provide an objective estimation, especially for HIEs. In this work, considering that the sit-to-stand (STS) exercise is one of the most implemented in physical rehabilitation, a computational model for estimating fatigue during this exercise is proposed. A study with 60 healthy volunteers was carried out to obtain a data set to develop and evaluate the proposed model. According to the literature, this model estimates three fatigue conditions (low, moderate, and high) by monitoring 32 STS kinematic features and the heart rate from a set of ambulatory sensors (Kinect and Zephyr sensors). Results show that a random forest model composed of 60 sub-classifiers presented an accuracy of 82.5% in the classification task. Moreover, results suggest that the movement of the upper body part is the most relevant feature for fatigue estimation. Movements of the lower body and the heart rate also contribute to essential information for identifying the fatigue condition. This work presents a promising tool for physical rehabilitation.

SVM classification of facial functions based on facial landmarks and animation Units

Quantitative assessment and classification of facial paralysis (FP) are essential for treatment selection and progress evaluation of the condition. As part of a comprehensive framework towards this goal, this study aims to classify five normal facial functions: smiling, eye closure, raising the eyebrows, blowing cheeks, and whistling as well as the rest state. 3D facial landmarks and facial animation units (FAUs) were obtained using the Kinect V2, a fast and cost-effective depth camera. These were used to compute the features used in a Support Vector Machine (SVM) classifier. A dataset of 1650 records from 50 normal subjects was compiled for this study. The performances of different SVM kernel models were tested with different feature groups. The best performance (Accuracy = 96.7%, Sensitivity = 90.2%, and Specificity = 98%) was found when using the RBF kernel model applied on just nine differences in FAUs. This research will be developed and extended to include FP classification.

Comparison of neuromuscular and cardiovascular exercise intensity and enjoyment between standard of care, off-the-shelf and custom active video games for promotion of physical activity of persons post-stroke

Background

Active video games have been embraced for the rehabilitation of mobility and promotion of physical activity for persons post-stroke. This study seeks to compare carefully matched standard of care stepping activities, off-the-shelf (non-custom) active video games and custom active video games that are either self-paced or game-paced for promoting neuromuscular intensity and accuracy, cardiovascular intensity, enjoyment and perceived effort.

Methods

Fifteen persons (ages 38–72) with mild to moderate severity in the chronic phase post-stroke (average 8 years) participated in a single group counter balanced repeated measures study. Participants were included if they were greater than 6 months post-stroke, who could walk 100 feet without assistance and stand unsupported for three continuous minutes. They were excluded if they had cardiac, musculoskeletal or neurologic conditions that could interfere with repeated stepping and follow instructions. In a single session located in a laboratory setting, participants executed for 8.5 min each: repeated stepping, the Kinect-light race game, two custom stepping games for the Kinect, one was repeated and self-paced and the other was random and game paced. Custom video games were adjusted to the participants stepping volume. Ten-minute rest periods followed the exercise during which time participants rested and completed the PACES an enjoyment questionnaire. Participants were instrumented with a metabolic cart and heart rate sensor for collection of cardiovascular intensity (METs and % of max HR) data. Stepping frequency, accuracy and pattern were acquired via video. Data were analyzed using a RMANOVA and post-hoc comparison with a Holm's/Sidak correction.

Results

Neuromuscular intensity (repetitions) was significantly greater for the off-the-shelf and self-paced custom game, however accuracy was greater for the custom games. Cardiovascular intensity for all activities took place in the moderate intensity exercise band. Enjoyment (measured with a questionnaire and rankings) was greater for the custom active video games and rate of perceived exertion was lower for the custom active video games.

Conclusions

Custom active video games provided comparable intensity but better accuracy, greater enjoyment and less perceived exertion than standard of care stepping activities and a carefully matched off-the-shelf (non-custom) video game. There were no differences between the game-paced and self-paced custom active video games.

Trial registration: NCT04538326.

Development and clinical evaluation of a contactless operating interface for three-dimensional image-guided navigation for endovascular neurosurgery

Purpose

In endovascular neurosurgery, the operator often acquires three-dimensional (3D) images of the cerebral vessels. Although workstation reoperation is required in some situations during treatment, it leads to time loss because a sterile condition cannot be maintained and treatment must be temporarily interrupted. Therefore, a workstation reoperating system is required while maintaining the desired sterility.

Methods

A contactless operating interface using Kinect to control 3D images was developed via gesture recognition for endovascular neurosurgery and was applied to a 3D volume rendering technique (VRT) image reconstructed at the workstation. The left-hand movement determines the assigned functions, whereas the right-hand movement is used like a computer mouse to pan and zoom in/out. In addition to the interface, voice commands were used and assigned to digital operations, such as image view changes and mode signal changes.

Results

This system was used for the actual endovascular treatment of cerebral aneurysms and cerebral arteriovenous malformations. The operator and gesture were recognized without any problems. Using voice operation, it was possible to expeditiously set the VRT image back to the reference angle. Furthermore, it was possible to finely adjust gesture operations, including mouse operation, and treatment was completed while maintaining sterile conditions.

Conclusion

A contactless operating interface was developed by combining the existing workstation system with Kinect and voice recognition software, allowing surgeons to perform a series of operations, which are normally performed in a console room, while maintaining sterile conditions.

Supplementary Information

The online version contains supplementary material available at 10.1007/s11548-021-02330-3.

Automatic Ankle Angle Detection by Integrated RGB and Depth Camera System

Depth cameras are developing widely. One of their main virtues is that, based on their data and by applying machine learning algorithms and techniques, it is possible to perform body tracking and make an accurate three-dimensional representation of body movement. Specifically, this paper will use the Kinect v2 device, which incorporates a random forest algorithm for 25 joints detection in the human body. However, although Kinect v2 is a powerful tool, there are circumstances in which the device's design does not allow the extraction of such data or the accuracy of the data is low, as is usually the case with foot position. We propose a method of acquiring this data in circumstances where the Kinect v2 device does not recognize the body when only the lower limbs are visible, improving the ankle angle's precision employing projection lines. Using a region-based convolutional neural network (Mask RCNN) for body recognition, raw data extraction for automatic ankle angle measurement has been achieved. All angles have been evaluated by inertial measurement units (IMUs) as gold standard. For the six tests carried out at different fixed distances between 0.5 and 4 m to the Kinect, we have obtained (mean ± SD) a Pearson's coefficient, r = 0.89 ± 0.04, a Spearman's coefficient, ρ = 0.83 ± 0.09, a root mean square error, RMSE = 10.7 ± 2.6 deg and a mean absolute error, MAE = 7.5 ± 1.8 deg. For the walking test, or variable distance test, we have obtained a Pearson's coefficient, r = 0.74, a Spearman's coefficient, ρ = 0.72, an RMSE = 6.4 deg and an MAE = 4.7 deg.

Simplified digital balance assessment in typically developing school children

BACKGROUND

Postural balance can be considered a conjoined parameter of gross motor performance. It is acquired in early childhood and honed until adolescence, but may also be influenced by various conditions. A simplified clinical assessment of balance and posture could be helpful in monitoring motor development or therapy particularly in pediatric patients. While analogue scales are considered unprecise and lab-based force-plate posturography lacks accessibility, we propose a novel kinematic balance assessment based on markerless 3D sensor technology.

RESEARCH QUESTION

Can balance and posture be assessed by tracking kinematic data using a single 3D motion tracking camera and are the results representative of normal motor development in a healthy pediatric cohort?

METHODS

A proprietary algorithm was developed and tested that uses skeletal data from the Microsoft Kinect™ V2 3D motion capture camera to calculate and track the center of mass in real time during a set of balance tasks. The algorithm tracks the distance of the COM traveled over time to calculate a balance score (COM speed). For this study, 432 school children aged 4-18 years performed 5 balance tasks and the resulting balance scores were analyzed and correlated with demographic data.

RESULTS

Preliminary experiments demonstrated that the system was able to reliably detect differences in COM speed during different balance tasks. The method showed moderate correlation with age and sex. Athletic activity positively correlated with balance skill in the age group < 8 years, but not in older children. Body mass appeared not to be correlated with balance ability.

SIGNIFICANCE

This study demonstrates that markerless 3D motion analysis can be used for the clinical assessment of coordination and balance and could potentially be used to monitor gross motor performance at the point-of-care.

Optimization of 3D Point Clouds of Oilseed Rape Plants Based on Time-of-Flight Cameras

Three-dimensional (3D) structure is an important morphological trait of plants for describing their growth and biotic/abiotic stress responses. Various methods have been developed for obtaining 3D plant data, but the data quality and equipment costs are the main factors limiting their development. Here, we propose a method to improve the quality of 3D plant data using the time-of-flight (TOF) camera Kinect V2. A K-dimension (k-d) tree was applied to spatial topological relationships for searching points. Background noise points were then removed with a minimum oriented bounding box (MOBB) with a pass-through filter, while outliers and flying pixel points were removed based on viewpoints and surface normals. After being smoothed with the bilateral filter, the 3D plant data were registered and meshed. We adjusted the mesh patches to eliminate layered points. The results showed that the patches were closer. The average distance between the patches was 1.88 × 10-3 m, and the average angle was 17.64°, which were 54.97% and 48.33% of those values before optimization. The proposed method performed better in reducing noise and the local layered-points phenomenon, and it could help to more accurately determine 3D structure parameters from point clouds and mesh models.

Evaluation of the Azure Kinect and Its Comparison to Kinect V1 and Kinect V2

The Azure Kinect is the successor of Kinect v1 and Kinect v2. In this paper we perform brief data analysis and comparison of all Kinect versions with focus on precision (repeatability) and various aspects of noise of these three sensors. Then we thoroughly evaluate the new Azure Kinect; namely its warm-up time, precision (and sources of its variability), accuracy (thoroughly, using a robotic arm), reflectivity (using 18 different materials), and the multipath and flying pixel phenomenon. Furthermore, we validate its performance in both indoor and outdoor environments, including direct and indirect sun conditions. We conclude with a discussion on its improvements in the context of the evolution of the Kinect sensor. It was shown that it is crucial to choose well designed experiments to measure accuracy, since the RGB and depth camera are not aligned. Our measurements confirm the officially stated values, namely standard deviation ≤17 mm, and distance error <11 mm in up to 3.5 meters distance from the sensor in all four supported modes. The device, however, has to be warmed up for at least 40-50 min to give stable results. Due to the time-of-flight technology, the Azure Kinect cannot be reliably used in direct sunlight. Therefore, it is convenient mostly for indoor applications.

Advanced Statistical Analysis of 3D Kinect Data: Mimetic Muscle Rehabilitation Following Head and Neck Surgeries Causing Facial Paresis

An advanced statistical analysis of patients’ faces after specific surgical procedures that temporarily negatively affect the patient’s mimetic muscles is presented. For effective planning of rehabilitation, which typically lasts several months, it is crucial to correctly evaluate the improvement of the mimetic muscle function. The current way of describing the development of rehabilitation depends on the subjective opinion and expertise of the clinician and is not very precise concerning when the most common classification (House–Brackmann scale) is used. Our system is based on a stereovision Kinect camera and an advanced mathematical approach that objectively quantifies the mimetic muscle function independently of the clinician’s opinion. To effectively deal with the complexity of the 3D camera input data and uncertainty of the evaluation process, we designed a three-stage data-analytic procedure combining the calculation of indicators determined by clinicians with advanced statistical methods including functional data analysis and ordinal (multiple) logistic regression. We worked with a dataset of 93 distinct patients and 122 sets of measurements. In comparison to the classification with the House–Brackmann scale the developed system is able to automatically monitor reinnervation of mimetic muscles giving us opportunity to discriminate even small improvements during the course of rehabilitation.

Comparative Analysis of Kinect-Based and Oculus-Based Gaze Region Estimation Methods in a Driving Simulator

Driver's gaze information can be crucial in driving research because of its relation to driver attention. Particularly, the inclusion of gaze data in driving simulators broadens the scope of research studies as they can relate drivers' gaze patterns to their features and performance. In this paper, we present two gaze region estimation modules integrated in a driving simulator. One uses the 3D Kinect device and another uses the virtual reality Oculus Rift device. The modules are able to detect the region, out of seven in which the driving scene was divided, where a driver is gazing at in every route processed frame. Four methods were implemented and compared for gaze estimation, which learn the relation between gaze displacement and head movement. Two are simpler and based on points that try to capture this relation and two are based on classifiers such as MLP and SVM. Experiments were carried out with 12 users that drove on the same scenario twice, each one with a different visualization display, first with a big screen and later with Oculus Rift. On the whole, Oculus Rift outperformed Kinect as the best hardware for gaze estimation. The Oculus-based gaze region estimation method with the highest performance achieved an accuracy of 97.94%. The information provided by the Oculus Rift module enriches the driving simulator data and makes it possible a multimodal driving performance analysis apart from the immersion and realism obtained with the virtual reality experience provided by Oculus.

The Reliability and Validity of Wearable Inertial Sensors Coupled with the Microsoft Kinect to Measure Shoulder Range-of-Motion

Background: Objective assessment of shoulder joint active range of motion (AROM) is critical to monitor patient progress after conservative or surgical intervention. Advancements in miniature devices have led researchers to validate inertial sensors to capture human movement. This study investigated the construct validity as well as intra- and inter-rater reliability of active shoulder mobility measurements using a coupled system of inertial sensors and the Microsoft Kinect (HumanTrak). Methods: 50 healthy participants with no history of shoulder pathology were tested bilaterally for fixed and free ROM: (1) shoulder flexion, and (2) abduction using HumanTrak and goniometry. The repeat testing of the standardised protocol was completed after seven days by two physiotherapists. Results: All HumanTrak shoulder movements demonstrated adequate reliability (intra-class correlation (ICC) ≥ 0.70). HumanTrak demonstrated higher intra-rater reliability (ICCs: 0.93 and 0.85) than goniometry (ICCs: 0.75 and 0.53) for measuring free shoulder flexion and abduction AROM, respectively. Similarly, HumanTrak demonstrated higher intra-rater reliability (ICCs: 0.81 and 0.94) than goniometry (ICCs: 0.70 and 0.93) for fixed flexion and abduction AROM, respectively. Construct validity between HumanTrak and goniometry was adequate except for free abduction. The differences between raters were predominately acceptable and below ±10°. Conclusions: These results indicated that the HumanTrak system is an objective, valid and reliable way to assess and track shoulder ROM.

The Validity and Reliability of the Microsoft Kinect for Measuring Trunk Compensation during Reaching

Compensatory movements at the trunk are commonly utilized during reaching by persons with motor impairments due to neurological injury such as stroke. Recent low-cost motion sensors may be able to measure trunk compensation, but their validity and reliability for this application are unknown. The purpose of this study was to compare the first (K1) and second (K2) generations of the Microsoft Kinect to a video motion capture system (VMC) for measuring trunk compensation during reaching. Healthy participants (n = 5) performed reaching movements designed to simulate trunk compensation in three different directions and on two different days while being measured by all three sensors simultaneously. Kinematic variables related to reaching range of motion (ROM), planar reach distance, trunk flexion and lateral flexion, shoulder flexion and lateral flexion, and elbow flexion were calculated. Validity and reliability were analyzed using repeated-measures ANOVA, paired t-tests, Pearson’s correlations, and Bland-Altman limits of agreement. Results show that the K2 was closer in magnitude to the VMC, more valid, and more reliable for measuring trunk flexion and lateral flexion during extended reaches than the K1. Both sensors were highly valid and reliable for reaching ROM, planar reach distance, and elbow flexion for all conditions. Results for shoulder flexion and abduction were mixed. The K2 was more valid and reliable for measuring trunk compensation during reaching and therefore might be prioritized for future development applications. Future analyses should include a more heterogeneous clinical population such as persons with chronic hemiparetic stroke.

Validity and sensitivity of instrumented postural and gait assessment using low-cost devices in Parkinson's disease

BACKGROUND

Accurate assessment of balance and gait is necessary to monitor the clinical progress of Parkinson's disease (PD). Conventional clinical scales can be biased and have limited accuracy. Novel interactive devices are potentially useful to detect subtle posture or gait-related impairments.

METHODS

Posturographic and single and dual-task gait assessments were performed to 54 individuals with PD and 43 healthy controls with the Wii Balance Board and the Kinect v2 and the, respectively. Individuals with PD were also assessed with the Tinetti Performance Oriented Mobility Assessment, the Functional Gait Assessment and the 10-m Walking Test. The influence of demographic and clinical variables on the performance in the instrumented posturographic and gait tests, the sensitivity of these tests to the clinical condition and phenotypes, and their convergent validity with clinical scales were investigated.

RESULTS

Individuals with PD in H&Y I and I.5 stages showed similar performance to controls. The greatest differences in posture and gait were found between subjects in H&Y II.5 and H&Y I-I.5 stage, as well as controls. Dual-tasking enhanced the differences among all groups in gait parameters. Akinetic/rigid phenotype showed worse postural control and gait than other phenotypes. High significant correlations were found between the limits of stability and most of gait parameters with the clinical scales.

CONCLUSIONS

Low-cost devices showed potential to objectively quantify posture and gait in established PD (H&Y ≥ II). Dual-tasking gait evaluation was more sensitive to detect differences among PD stages and compared to controls than free gait. Gait and posture were more impaired in akinetic/rigid PD.