Statistical Validation for Clinical Measures: Repeatability and Agreement of Kinect™-Based Software

Scoring People With Spinal Muscular Atrophy on the Motor Function Measure Using the Microsoft Kinect

PURPOSE

Assess the ability of the Kinect to capture movement and posture of people with spinal muscular atrophy (SMA) during completion of 14 items of the Motor Function Measure, a validated functional rating scale for people with neuromuscular diseases.

METHODS

Multicenter feasibility study in which Motor Function Measure items were scored as usual by the participant's therapist during the completion (Score-T) while another therapist scored items based only on the visualization of digital data collected using the Kinect (Score-D). Agreement and disagreement were investigated.

RESULTS

Twenty people with SMA type 2 or 3 were participants; 142 items were recorded and analyzed. There was 31.7% agreement between Score-T and Score-D for participants with SMA type 2, and 76.2% for those with SMA type 3.

CONCLUSIONS

The results prevent us from considering the use of Kinect capture to deduce an automated scoring, but this device may be of interest to highlight potential compensations.

Reliability of 3D Depth Motion Sensors for Capturing Upper Body Motions and Assessing the Quality of Wheelchair Transfers

Wheelchair users must use proper technique when performing sitting-pivot-transfers (SPTs) to prevent upper extremity pain and discomfort. Current methods to analyze the quality of SPTs include the TransKinect, a combination of machine learning (ML) models, and the Transfer Assessment Instrument (TAI), to automatically score the quality of a transfer using Microsoft Kinect V2. With the discontinuation of the V2, there is a necessity to determine the compatibility of other commercial sensors. The Intel RealSense D435 and the Microsoft Kinect Azure were compared against the V2 for inter- and intra-sensor reliability. A secondary analysis with the Azure was also performed to analyze its performance with the existing ML models used to predict transfer quality. The intra- and inter-sensor reliability was higher for the Azure and V2 (n = 7; ICC = 0.63 to 0.92) than the RealSense and V2 (n = 30; ICC = 0.13 to 0.7) for four key features. Additionally, the V2 and the Azure both showed high agreement with each other on the ML outcomes but not against a ground truth. Therefore, the ML models may need to be retrained ideally with the Azure, as it was found to be a more reliable and robust sensor for tracking wheelchair transfers in comparison to the V2.

The Reliability of the Microsoft Kinect and Ambulatory Sensor-Based Motion Tracking Devices to Measure Shoulder Range-of-Motion: A Systematic Review and Meta-Analysis

Advancements in motion sensing technology can potentially allow clinicians to make more accurate range-of-motion (ROM) measurements and informed decisions regarding patient management. The aim of this study was to systematically review and appraise the literature on the reliability of the Kinect, inertial sensors, smartphone applications and digital inclinometers/goniometers to measure shoulder ROM. Eleven databases were screened (MEDLINE, EMBASE, EMCARE, CINAHL, SPORTSDiscus, Compendex, IEEE Xplore, Web of Science, Proquest Science and Technology, Scopus, and PubMed). The methodological quality of the studies was assessed using the consensus-based standards for the selection of health Measurement Instruments (COSMIN) checklist. Reliability assessment used intra-class correlation coefficients (ICCs) and the criteria from Swinkels et al. (2005). Thirty-two studies were included. A total of 24 studies scored "adequate" and 2 scored "very good" for the reliability standards. Only one study scored "very good" and just over half of the studies (18/32) scored "adequate" for the measurement error standards. Good intra-rater reliability (ICC > 0.85) and inter-rater reliability (ICC > 0.80) was demonstrated with the Kinect, smartphone applications and digital inclinometers. Overall, the Kinect and ambulatory sensor-based human motion tracking devices demonstrate moderate-good levels of intra- and inter-rater reliability to measure shoulder ROM. Future reliability studies should focus on improving study design with larger sample sizes and recommended time intervals between repeated measurements.

Baseline, retest, and post-injury profiles of auditory neural function in collegiate football players

OBJECTIVES

Recent retrospective studies report differences in auditory neurophysiology between concussed athletes and uninjured controls using the frequency-following response (FFR). Adopting a prospective design in college football players, we compared FFRs before and after a concussion and evaluated test-retest reliability in non-concussed teammates.

DESIGN

Testing took place in a locker room. We analysed the FFR to the fundamental frequency (F0) (FFR-F0) of a speech stimulus, previously identified as a potential concussion biomarker. Baseline FFRs were obtained during the football pre-season. In athletes diagnosed with concussions during the season, FFRs were measured days after injury and compared to pre-season baseline. In uninjured controls, comparisons were made between pre- and post-season.

STUDY SAMPLE

Participants were Tulane University football athletes (n = 65).

RESULTS

In concussed athletes, there was a significant group-level decrease in FFR-F0 from baseline (26% decrease on average). By contrast, the control group's change from baseline was not statistically significant, and comparisons of pre- and post-season had good repeatability (intraclass correlation coefficient = 0.75).

CONCLUSIONS

Results converge with previous work to evince suppressed neural function to the FFR-F0 following concussion. This preliminary study paves the way for larger-scale clinical evaluation of the specificity and reliability of the FFR as a concussion diagnostic.HighlightsThis prospective study reveals suppressed neural responses to sound in concussed athletes compared to baseline.Neural responses to sound show good repeatability in uninjured athletes tested in a locker-room setting.Results support the feasibility of recording frequency-following responses in non-laboratory conditions.

The Reliability and Validity of Wearable Inertial Sensors Coupled with the Microsoft Kinect to Measure Shoulder Range-of-Motion

Background: Objective assessment of shoulder joint active range of motion (AROM) is critical to monitor patient progress after conservative or surgical intervention. Advancements in miniature devices have led researchers to validate inertial sensors to capture human movement. This study investigated the construct validity as well as intra- and inter-rater reliability of active shoulder mobility measurements using a coupled system of inertial sensors and the Microsoft Kinect (HumanTrak). Methods: 50 healthy participants with no history of shoulder pathology were tested bilaterally for fixed and free ROM: (1) shoulder flexion, and (2) abduction using HumanTrak and goniometry. The repeat testing of the standardised protocol was completed after seven days by two physiotherapists. Results: All HumanTrak shoulder movements demonstrated adequate reliability (intra-class correlation (ICC) ≥ 0.70). HumanTrak demonstrated higher intra-rater reliability (ICCs: 0.93 and 0.85) than goniometry (ICCs: 0.75 and 0.53) for measuring free shoulder flexion and abduction AROM, respectively. Similarly, HumanTrak demonstrated higher intra-rater reliability (ICCs: 0.81 and 0.94) than goniometry (ICCs: 0.70 and 0.93) for fixed flexion and abduction AROM, respectively. Construct validity between HumanTrak and goniometry was adequate except for free abduction. The differences between raters were predominately acceptable and below ±10°. Conclusions: These results indicated that the HumanTrak system is an objective, valid and reliable way to assess and track shoulder ROM.