Vision based interface: an alternative tool for children with cerebral palsy

Computer Vision-based Methodology to Improve Interaction for People with Motor and Speech Impairment

Augmentative and Alternative Communication (AAC) aims to complement or replace spoken language to compensate for expression difficulties faced by people with speech impairments. Computing systems have been developed to support AAC; however, partially due to technical problems, poor interface, and limited interaction functions, AAC systems are not widespread, adopted, and used, therefore reaching a limited audience. This article proposes a methodology to support AAC for people with motor impairments, using computer vision and machine learning techniques to allow for personalized gestural interaction. The methodology was applied in a pilot system used by both volunteers without disabilities, and by volunteers with motor and speech impairments, to create datasets with personalized gestures. The created datasets and a public dataset were used to evaluate the technologies employed for gesture recognition, namely the Support Vector Machine (SVM) and Convolutional Neural Network (using Transfer Learning), and for motion representation, namely the conventional Motion History Image and Optical Flow-Motion History Image (OF-MHI). Results obtained from the estimation of prediction error using K-fold cross-validation suggest SVM associated with OF-MHI presents slightly better results for gesture recognition. Results indicate the technical feasibility of the proposed methodology, which uses a low-cost approach, and reveals the challenges and specific needs observed during the experiment with the target audience.

Statistical Validation for Clinical Measures: Repeatability and Agreement of Kinect™-Based Software

Background

The rehabilitation process is a fundamental stage for recovery of people's capabilities. However, the evaluation of the process is performed by physiatrists and medical doctors, mostly based on their observations, that is, a subjective appreciation of the patient's evolution. This paper proposes a tracking platform of the movement made by an individual's upper limb using Kinect sensor(s) to be applied for the patient during the rehabilitation process. The main contribution is the development of quantifying software and the statistical validation of its performance, repeatability, and clinical use in the rehabilitation process.

Methods

The software determines joint angles and upper limb trajectories for the construction of a specific rehabilitation protocol and quantifies the treatment evolution. In turn, the information is presented via a graphical interface that allows the recording, storage, and report of the patient's data. For clinical purposes, the software information is statistically validated with three different methodologies, comparing the measures with a goniometer in terms of agreement and repeatability.

Results

The agreement of joint angles measured with the proposed software and goniometer is evaluated with Bland-Altman plots; all measurements fell well within the limits of agreement, meaning interchangeability of both techniques. Additionally, the results of Bland-Altman analysis of repeatability show 95% confidence. Finally, the physiotherapists' qualitative assessment shows encouraging results for the clinical use.

Conclusion

The main conclusion is that the software is capable of offering a clinical history of the patient and is useful for quantification of the rehabilitation success. The simplicity, low cost, and visualization possibilities enhance the use of the software Kinect for rehabilitation and other applications, and the expert's opinion endorses the choice of our approach for clinical practice. Comparison of the new measurement technique with established goniometric methods determines that the proposed software agrees sufficiently to be used interchangeably.