1
|
Collings TJ, Devaprakash D, Pizzolato C, Lloyd DG, Barrett RS, Lenton GK, Thomeer LT, Bourne MN. Inclusion of a skeletal model partly improves the reliability of lower limb joint angles derived from a markerless depth camera. J Biomech 2024; 170:112160. [PMID: 38824704 DOI: 10.1016/j.jbiomech.2024.112160] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/20/2023] [Revised: 03/19/2024] [Accepted: 05/20/2024] [Indexed: 06/04/2024]
Abstract
A single depth camera provides a fast and easy approach to performing biomechanical assessments in a clinical setting; however, there are currently no established methods to reliably determine joint angles from these devices. The primary aim of this study was to compare joint angles as well as the between-day reliability of direct kinematics to model-constrained inverse kinematics recorded using a single markerless depth camera during a range of clinical and athletic movement assessments.A secondary aim was to determine the minimum number of trials required to maximize reliability. Eighteen healthy participants attended two testing sessions one week apart. Tasks included treadmill walking, treadmill running, single-leg squats, single-leg countermovement jumps, bilateral countermovement jumps, and drop vertical jumps. Keypoint data were processed using direct kinematics as well as in OpenSim using a full-body musculoskeletal model and inverse kinematics. Kinematic methods were compared using statistical parametric mapping and between-day reliability was calculated using intraclass correlation coefficients, mean absolute error, and minimal detectable change. Keypoint-derived inverse kinematics resulted in significantly smaller hip flexion (range = -9 to -2°), hip abduction (range = -3 to -2°), knee flexion (range = -5° to -2°), and greater dorsiflexion angles (range = 6-15°) than direct kinematics. Both markerless kinematic methods had high between-day reliability (inverse kinematics ICC 95 %CI = 0.83-0.90; direct kinematics ICC 95 %CI = 0.80-0.93). For certain tasks and joints, keypoint-derived inverse kinematics resulted in greater reliability (up to 0.47 ICC) and smaller minimal detectable changes (up to 13°) than direct kinematics. Performing 2-4 trials was sufficient to maximize reliability for most tasks. A single markerless depth camera can reliably measure lower limb joint angles, and skeletal model-constrained inverse kinematics improves lower limb joint angle reliability for certain tasks and joints.
Collapse
Affiliation(s)
- Tyler J Collings
- School of Health Sciences and Social Work, Griffith University, Gold Coast Campus, Australia; Griffith Centre of Biomedical and Rehabilitation Engineering (GCORE), In The Menzies Health Institute Queensland and Advanced Design and Prototyping Technologies Institute, Griffith University, Gold Coast Campus, Australia.
| | - Daniel Devaprakash
- School of Health Sciences and Social Work, Griffith University, Gold Coast Campus, Australia; Griffith Centre of Biomedical and Rehabilitation Engineering (GCORE), In The Menzies Health Institute Queensland and Advanced Design and Prototyping Technologies Institute, Griffith University, Gold Coast Campus, Australia; Vald Performance, Brisbane, Queensland, Australia
| | - Claudio Pizzolato
- School of Health Sciences and Social Work, Griffith University, Gold Coast Campus, Australia; Griffith Centre of Biomedical and Rehabilitation Engineering (GCORE), In The Menzies Health Institute Queensland and Advanced Design and Prototyping Technologies Institute, Griffith University, Gold Coast Campus, Australia
| | - David G Lloyd
- School of Health Sciences and Social Work, Griffith University, Gold Coast Campus, Australia; Griffith Centre of Biomedical and Rehabilitation Engineering (GCORE), In The Menzies Health Institute Queensland and Advanced Design and Prototyping Technologies Institute, Griffith University, Gold Coast Campus, Australia
| | - Rod S Barrett
- School of Health Sciences and Social Work, Griffith University, Gold Coast Campus, Australia; Griffith Centre of Biomedical and Rehabilitation Engineering (GCORE), In The Menzies Health Institute Queensland and Advanced Design and Prototyping Technologies Institute, Griffith University, Gold Coast Campus, Australia
| | | | | | - Matthew N Bourne
- School of Health Sciences and Social Work, Griffith University, Gold Coast Campus, Australia; Griffith Centre of Biomedical and Rehabilitation Engineering (GCORE), In The Menzies Health Institute Queensland and Advanced Design and Prototyping Technologies Institute, Griffith University, Gold Coast Campus, Australia
| |
Collapse
|
2
|
Furtado S, Galna B, Godfrey A, Rochester L, Gerrand C. Feasibility of using low-cost markerless motion capture for assessing functional outcomes after lower extremity musculoskeletal cancer surgery. PLoS One 2024; 19:e0300351. [PMID: 38547229 PMCID: PMC10977781 DOI: 10.1371/journal.pone.0300351] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/02/2023] [Accepted: 02/26/2024] [Indexed: 04/02/2024] Open
Abstract
BACKGROUND Physical limitations are frequent and debilitating after sarcoma treatment. Markerless motion capture (MMC) could measure these limitations. Historically expensive cumbersome systems have posed barriers to clinical translation. RESEARCH QUESTION Can inexpensive MMC [using Microsoft KinectTM] assess functional outcomes after sarcoma surgery, discriminate between tumour sub-groups and agree with existing assessments? METHODS Walking, unilateral stance and kneeling were measured in a cross-sectional study of patients with lower extremity sarcomas using MMC and standard video. Summary measures of temporal, balance, gait and movement velocity were derived. Feasibility and early indicators of validity of MMC were explored by comparing MMC measures i) between tumour sub-groups; ii) against video and iii) with established sarcoma tools [Toronto Extremity Salvage Score (TESS)), Musculoskeletal Tumour Rating System (MSTS), Quality of life-cancer survivors (QoL-CS)]. Statistical analysis was conducted using SPSS v19. Tumour sub-groups were compared using Mann-Whitney U tests, MMC was compared to existing sarcoma measures using correlations and with video using Intraclass correlation coefficient agreement. RESULTS Thirty-four adults of mean age 43 (minimum value-maximum value 19-89) years with musculoskeletal tumours in the femur (19), pelvis/hip (3), tibia (9), or ankle/foot (3) participated; 27 had limb sparing surgery and 7 amputation. MMC was well-tolerated and feasible to deliver. MMC discriminated between surgery groups for balance (p<0.05*), agreed with video for kneeling times [ICC = 0.742; p = 0.001*] and showed moderate relationships between MSTS and gait (p = 0.022*, r = -0.416); TESS and temporal outcomes (p = 0.016* and r = -0.0557*), movement velocity (p = 0.021*, r = -0.541); QoL-CS and balance (p = 0.027*, r = 0.441) [* = statistical significance]. As MMC uncovered important relationships between outcomes, it gave an insight into how functional impairments, balance, gait, disabilities and quality of life (QoL) are associated with each other. This gives an insight into mechanisms of poor outcomes, producing clinically useful data i.e. data which can inform clinical practice and guide the delivery of targeted rehabilitation. For example, patients presenting with poor balance in various activities can be prescribed with balance rehabilitation and those with difficulty in movements or activity transitions can be managed with exercises and training to improve the quality and efficiency of the movement. SIGNIFICANCE In this first study world-wide, investigating the use of MMC after sarcoma surgery, MMC was found to be acceptable and feasible to assess functional outcomes in this cancer population. MMC demonstrated early indicators of validity and also provided new knowledge that functional impairments are related to balance during unilateral stance and kneeling, gait and movement velocity during kneeling and these outcomes in turn are related to disabilities and QoL. This highlighted important relationships between different functional outcomes and QoL, providing valuable information for delivering personalised rehabilitation. After completing future validation work in a larger study, this approach can offer promise in clinical settings. Low-cost MMC shows promise in assessing patient's impairments in the hospitals or their homes and guiding clinical management and targeted rehabilitation based on novel MMC outcomes affected, therefore providing an opportunity for delivering personalised exercise programmes and physiotherapy care delivery for this rare cancer.
Collapse
Affiliation(s)
- Sherron Furtado
- Department of Orthopaedics and Musculoskeletal Science, University College London, London, United Kingdom
- Therapies and Department of Orthopaedic Oncology, London Sarcoma Service, Royal National Orthopaedic Hospital NHS Trust, Stanmore, United Kingdom
| | - Brook Galna
- School of Allied Health (Exercise Science), Murdoch University, Perth, Australia
- Translational and Clinical Research Institute, Newcastle University, Newcastle upon Tyne, United Kingdom
| | - Alan Godfrey
- Computer and Information Science Department, Northumbria University, Newcastle upon Tyne, United Kingdom
| | - Lynn Rochester
- Translational and Clinical Research Institute, Newcastle University, Newcastle upon Tyne, United Kingdom
- Newcastle upon Tyne Hospitals NHS Foundation Trust, Newcastle upon Tyne, United Kingdom
| | - Craig Gerrand
- Department of Orthopaedic Oncology, The London Sarcoma Service, Royal National Orthopaedic Hospital NHS Trust, Stanmore, United Kingdom
| |
Collapse
|
3
|
Wang Y, Tang R, Wang H, Yu X, Li Y, Wang C, Wang L, Qie S. The Validity and Reliability of a New Intelligent Three-Dimensional Gait Analysis System in Healthy Subjects and Patients with Post-Stroke. SENSORS (BASEL, SWITZERLAND) 2022; 22:9425. [PMID: 36502143 PMCID: PMC9740023 DOI: 10.3390/s22239425] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 10/21/2022] [Revised: 11/28/2022] [Accepted: 11/29/2022] [Indexed: 06/17/2023]
Abstract
Odonate is a new, intelligent three-dimensional gait analysis system based on binocular depth cameras and neural networks, but its accuracy has not been validated. Twenty-six healthy subjects and sixteen patients with post-stroke were recruited to investigate the validity and reliability of Odonate for gait analysis and examine its ability to discriminate abnormal gait patterns. The repeatability tests of different raters and different days showed great consistency. Compared with the results measured by Vicon, gait velocity, cadence, step length, cycle time, and sagittal hip and knee joint angles measured by Odonate showed high consistency, while the consistency of the gait phase division and the sagittal ankle joint angle was slightly lower. In addition, the stages with statistical differences between healthy subjects and patients during a gait cycle measured by the two systems were consistent. In conclusion, Odonate has excellent inter/intra-rater reliability, and has strong validity in measuring some spatiotemporal parameters and the sagittal joint angles, except the gait phase division and the ankle joint angle. Odonate is comparable to Vicon in its ability to identify abnormal gait patterns in patients with post-stroke. Therefore, Odonate has the potential to provide accessible and objective measurements for clinical gait assessment.
Collapse
Affiliation(s)
- Yingpeng Wang
- Department of Rehabilitation, Beijing Rehabilitation Hospital, Capital Medical University, Beijing 100144, China
| | - Ran Tang
- Department of Rehabilitation, Beijing Rehabilitation Hospital, Capital Medical University, Beijing 100144, China
| | - Hujun Wang
- Department of Rehabilitation, Beijing Rehabilitation Hospital, Capital Medical University, Beijing 100144, China
| | - Xin Yu
- Beijing Rehabilitation Medical College, Capital Medical University, Beijing 100144, China
| | - Yingqi Li
- Department of Rehabilitation, Beijing Rehabilitation Hospital, Capital Medical University, Beijing 100144, China
| | - Congxiao Wang
- Department of Rehabilitation, Beijing Rehabilitation Hospital, Capital Medical University, Beijing 100144, China
| | - Luyi Wang
- Department of Rehabilitation, Beijing Rehabilitation Hospital, Capital Medical University, Beijing 100144, China
| | - Shuyan Qie
- Department of Rehabilitation, Beijing Rehabilitation Hospital, Capital Medical University, Beijing 100144, China
| |
Collapse
|
4
|
Liu PL, Chang CC, Li L, Xu X. A Simple Method to Optimally Select Upper-Limb Joint Angle Trajectories from Two Kinect Sensors during the Twisting Task for Posture Analysis. SENSORS (BASEL, SWITZERLAND) 2022; 22:s22197662. [PMID: 36236761 PMCID: PMC9572104 DOI: 10.3390/s22197662] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/19/2022] [Revised: 09/26/2022] [Accepted: 10/06/2022] [Indexed: 05/17/2023]
Abstract
A trunk-twisting posture is strongly associated with physical discomfort. Measurement of joint kinematics to assess physical exposure to injuries is important. However, using a single Kinect sensor to track the upper-limb joint angle trajectories during twisting tasks in the workplace is challenging due to sensor view occlusions. This study provides and validates a simple method to optimally select the upper-limb joint angle data from two Kinect sensors at different viewing angles during the twisting task, so the errors of trajectory estimation can be improved. Twelve healthy participants performed a rightward twisting task. The tracking errors of the upper-limb joint angle trajectories of two Kinect sensors during the twisting task were estimated based on concurrent data collected using a conventional motion tracking system. The error values were applied to generate the error trendlines of two Kinect sensors using third-order polynomial regressions. The intersections between two error trendlines were used to define the optimal data selection points for data integration. The finding indicates that integrating the outputs from two Kinect sensor datasets using the proposed method can be more robust than using a single sensor for upper-limb joint angle trajectory estimations during the twisting task.
Collapse
Affiliation(s)
- Pin-Ling Liu
- Department of Industrial Engineering and Engineering Management, National Tsing Hua University, Hsinchu 300, Taiwan
| | - Chien-Chi Chang
- Department of Industrial Engineering and Engineering Management, National Tsing Hua University, Hsinchu 300, Taiwan
- Correspondence: ; Tel.: +886-3-5742942
| | - Li Li
- Edward P. Fitts Department of Industrial and Systems Engineering, North Carolina State University, Raleigh, NC 27695, USA
| | - Xu Xu
- Edward P. Fitts Department of Industrial and Systems Engineering, North Carolina State University, Raleigh, NC 27695, USA
| |
Collapse
|
5
|
Oh J, Ripic Z, Signorile JF, Andersen MS, Kuenze C, Letter M, Best TM, Eltoukhy M. Monitoring joint mechanics in anterior cruciate ligament reconstruction using depth sensor-driven musculoskeletal modeling and statistical parametric mapping. Med Eng Phys 2022; 103:103796. [DOI: 10.1016/j.medengphy.2022.103796] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/27/2022] [Revised: 03/04/2022] [Accepted: 04/05/2022] [Indexed: 11/28/2022]
|
6
|
Hu G, Wang W, Chen B, Zhi H, Yudi Li, Shen Y, Wang K. Concurrent validity of evaluating knee kinematics using Kinect system during rehabilitation exercise. MEDICINE IN NOVEL TECHNOLOGY AND DEVICES 2021. [DOI: 10.1016/j.medntd.2021.100068] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/20/2022] Open
|
7
|
Liu PL, Chang CC, Lin JH, Kobayashi Y. Simple benchmarking method for determining the accuracy of depth cameras in body landmark location estimation: Static upright posture as a measurement example. PLoS One 2021; 16:e0254814. [PMID: 34288917 PMCID: PMC8294549 DOI: 10.1371/journal.pone.0254814] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/19/2021] [Accepted: 07/04/2021] [Indexed: 11/19/2022] Open
Abstract
To evaluate the postures in ergonomics applications, studies have proposed the use of low-cost, marker-less, and portable depth camera-based motion tracking systems (DCMTSs) as a potential alternative to conventional marker-based motion tracking systems (MMTSs). However, a simple but systematic method for examining the estimation errors of various DCMTSs is lacking. This paper proposes a benchmarking method for assessing the estimation accuracy of depth cameras for full-body landmark location estimation. A novel alignment board was fabricated to align the coordinate systems of the DCMTSs and MMTSs. The data from an MMTS were used as a reference to quantify the error of using a DCMTS to identify target locations in a 3-D space. To demonstrate the proposed method, the full-body landmark location tracking errors were evaluated for a static upright posture using two different DCMTSs. For each landmark, we compared each DCMTS (Kinect system and RealSense system) with an MMTS by calculating the Euclidean distances between symmetrical landmarks. The evaluation trials were performed twice. The agreement between the tracking errors of the two evaluation trials was assessed using intraclass correlation coefficient (ICC). The results indicate that the proposed method can effectively assess the tracking performance of DCMTSs. The average errors (standard deviation) for the Kinect system and RealSense system were 2.80 (1.03) cm and 5.14 (1.49) cm, respectively. The highest average error values were observed in the depth orientation for both DCMTSs. The proposed method achieved high reliability with ICCs of 0.97 and 0.92 for the Kinect system and RealSense system, respectively.
Collapse
Affiliation(s)
- Pin-Ling Liu
- Department of Industrial Engineering and Engineering Management, National Tsing Hua University, Hsinchu, Taiwan
| | - Chien-Chi Chang
- Department of Industrial Engineering and Engineering Management, National Tsing Hua University, Hsinchu, Taiwan
- * E-mail:
| | - Jia-Hua Lin
- Washington State Department of Labor and Industries, Olympia, Washington, United States of America
| | - Yoshiyuki Kobayashi
- Human Augmentation Research Center, National Institute of Advanced Industrial Science and Technology, Tokyo, Japan
| |
Collapse
|
8
|
Vilas-Boas MDC, Rocha AP, Cardoso MN, Fernandes JM, Coelho T, Cunha JPS. Supporting the Assessment of Hereditary Transthyretin Amyloidosis Patients Based On 3-D Gait Analysis and Machine Learning. IEEE Trans Neural Syst Rehabil Eng 2021; 29:1350-1362. [PMID: 34252029 DOI: 10.1109/tnsre.2021.3096433] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
Hereditary Transthyretin Amyloidosis (vATTR-V30M) is a rare and highly incapacitating sensorimotor neuropathy caused by an inherited mutation (Val30Met), which typically affects gait, among other symptoms. In this context, we investigated the possibility of using machine learning (ML) techniques to build a model(s) that can be used to support the detection of the Val30Met mutation (possibility of developing the disease), as well as symptom onset detection for the disease, given the gait characteristics of a person. These characteristics correspond to 24 gait parameters computed from 3-D body data, provided by a Kinect v2 camera, acquired from a person while walking towards the camera. To build the model(s), different ML algorithms were explored: k-nearest neighbors, decision tree, random forest, support vector machines (SVM), and multilayer perceptron. For a dataset corresponding to 66 subjects (25 healthy controls, 14 asymptomatic mutation carriers, and 27 patients) and several gait cycles per subject, we were able to obtain a model that distinguishes between controls and vATTR-V30M mutation carriers (with or without symptoms) with a mean accuracy of 92% (SVM). We also obtained a model that distinguishes between asymptomatic and symptomatic carriers with a mean accuracy of 98% (SVM). These results are very relevant, since this is the first study that proposes a ML approach to support vATTR-V30M patient assessment based on gait, being a promising foundation for the development of a computer-aided diagnosis tool to help clinicians in the identification and follow-up of this disease. Furthermore, the proposed method may also be used for other neuropathies.
Collapse
|
9
|
Kinect V2-Based Gait Analysis for Children with Cerebral Palsy: Validity and Reliability of Spatial Margin of Stability and Spatiotemporal Variables. SENSORS 2021; 21:s21062104. [PMID: 33802731 PMCID: PMC8002565 DOI: 10.3390/s21062104] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 12/16/2020] [Revised: 01/15/2021] [Accepted: 03/13/2021] [Indexed: 12/17/2022]
Abstract
Children with cerebral palsy (CP) have high risks of falling. It is necessary to evaluate gait stability for children with CP. In comparison to traditional motion capture techniques, the Kinect has the potential to be utilised as a cost-effective gait stability assessment tool, ensuring frequent and uninterrupted gait monitoring. To evaluate the validity and reliability of this measurement, in this study, ten children with CP performed two testing sessions, of which gait data were recorded by a Kinect V2 sensor and a referential Motion Analysis system. The margin of stability (MOS) and gait spatiotemporal metrics were examined. For the spatiotemporal parameters, intraclass correlation coefficient (ICC2,k) values were from 0.83 to 0.99 between two devices and from 0.78 to 0.88 between two testing sessions. For the MOS outcomes, ICC2,k values ranged from 0.42 to 0.99 between two devices and 0.28 to 0.69 between two test sessions. The Kinect V2 was able to provide valid and reliable spatiotemporal gait parameters, and it could also offer accurate outcome measures for the minimum MOS. The reliability of the Kinect V2 when assessing time-specific MOS variables was limited. The Kinect V2 shows the potential to be used as a cost-effective tool for CP gait stability assessment.
Collapse
|
10
|
Díaz-San Martín G, Reyes-González L, Sainz-Ruiz S, Rodríguez-Cobo L, López-Higuera JM. Automatic Ankle Angle Detection by Integrated RGB and Depth Camera System. SENSORS (BASEL, SWITZERLAND) 2021; 21:1909. [PMID: 33803369 PMCID: PMC7967151 DOI: 10.3390/s21051909] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 01/14/2021] [Revised: 02/26/2021] [Accepted: 03/02/2021] [Indexed: 11/16/2022]
Abstract
Depth cameras are developing widely. One of their main virtues is that, based on their data and by applying machine learning algorithms and techniques, it is possible to perform body tracking and make an accurate three-dimensional representation of body movement. Specifically, this paper will use the Kinect v2 device, which incorporates a random forest algorithm for 25 joints detection in the human body. However, although Kinect v2 is a powerful tool, there are circumstances in which the device's design does not allow the extraction of such data or the accuracy of the data is low, as is usually the case with foot position. We propose a method of acquiring this data in circumstances where the Kinect v2 device does not recognize the body when only the lower limbs are visible, improving the ankle angle's precision employing projection lines. Using a region-based convolutional neural network (Mask RCNN) for body recognition, raw data extraction for automatic ankle angle measurement has been achieved. All angles have been evaluated by inertial measurement units (IMUs) as gold standard. For the six tests carried out at different fixed distances between 0.5 and 4 m to the Kinect, we have obtained (mean ± SD) a Pearson's coefficient, r = 0.89 ± 0.04, a Spearman's coefficient, ρ = 0.83 ± 0.09, a root mean square error, RMSE = 10.7 ± 2.6 deg and a mean absolute error, MAE = 7.5 ± 1.8 deg. For the walking test, or variable distance test, we have obtained a Pearson's coefficient, r = 0.74, a Spearman's coefficient, ρ = 0.72, an RMSE = 6.4 deg and an MAE = 4.7 deg.
Collapse
Affiliation(s)
- Guillermo Díaz-San Martín
- Photonics Engineering Group, University of Cantabria, 39005 Santander, Spain; (L.R.-G.); (S.S.-R.); (J.M.L.-H.)
| | - Luis Reyes-González
- Photonics Engineering Group, University of Cantabria, 39005 Santander, Spain; (L.R.-G.); (S.S.-R.); (J.M.L.-H.)
| | - Sergio Sainz-Ruiz
- Photonics Engineering Group, University of Cantabria, 39005 Santander, Spain; (L.R.-G.); (S.S.-R.); (J.M.L.-H.)
| | | | - José M. López-Higuera
- Photonics Engineering Group, University of Cantabria, 39005 Santander, Spain; (L.R.-G.); (S.S.-R.); (J.M.L.-H.)
- CIBER-bbn, Instituto de Salud Carlos III, 28029 Madrid, Spain;
- Instituto de Investigación Sanitaria Valdecilla (IDIVAL), 39011 Santander, Spain
| |
Collapse
|
11
|
Zhu Y, Lu W, Gan W, Hou W. A contactless method to measure real-time finger motion using depth-based pose estimation. Comput Biol Med 2021; 131:104282. [PMID: 33631496 DOI: 10.1016/j.compbiomed.2021.104282] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/17/2021] [Revised: 02/11/2021] [Accepted: 02/11/2021] [Indexed: 10/22/2022]
Abstract
BACKGROUND Finger mobility plays a crucial role in everyday living and is a leading indicator during hand rehabilitation and assistance tasks. Depth-based hand pose estimation is a potentially low-cost solution for the clinical and home-based measurement of symptoms of limited human finger motion. OBJECTIVE The purpose of this study was to achieve the contactless measurement of finger motion based on depth-based hand pose estimation using Azure Kinect depth cameras and transfer learning, and to evaluate the accuracy in comparison with a three-dimensional motion analysis (3DMA) system. METHODS Thirty participants performed a series of tasks during which their hand motions were measured concurrently using the Azure Kinect and 3DMA systems. We propose a simple and effective approach to achieving real-time hand pose estimations from single depth images using ensemble convolutional neural networks trained by a transfer learning strategy. Algorithms to calculate the finger joint motion angles are presented by tracking the locations of the 24 hand joints. To demonstrate their potential, the Azure-Kinect-based 3D finger motion measurement system and algorithms are experimentally verified through comparison with a camera-based 3DMA system, which is the gold standard. RESULTS Our results revealed that the Azure-Kinect-based hand pose estimation system produced highly correlated measurements of hand joint coordinates. Our method achieved excellent performance in terms of the fraction of good frames ( >80%) when the error thresholds were larger than approximately 2 cm, and the range of mean error distance was 0.23--1.05 cm. For joint angles, the Azure Kinect and 3DMA systems had comparable inter-trial reliability (ICC2,1 ranging from 0.89 to 0.97) and excellent concurrent validity, with Pearsons r-values >0.90 for most measurements (range: 0.88--0.97). The 95% BlandAltman limits of agreement were narrow enough for the Azure Kinect to be considered a valid tool for the measurement of all reported joint angles of the index finger and thumb in pinching. Moreover, our method runs in real time at over 45 fps. CONCLUSION The results of this study suggest that the proposed method has the capacity to measure the performance of fine motor skills.
Collapse
Affiliation(s)
- Yean Zhu
- Bioengineering College of Chongqing University, Chongqing, China; School of Transportation and Logistics, East China Jiaotong University, Nanchang, China.
| | - Wei Lu
- Department of Rehabilitation Medicine, Jiangxi Provincial Peoples Hospital, Nanchang, China.
| | - Weihua Gan
- School of Transportation and Logistics, East China Jiaotong University, Nanchang, China.
| | - Wensheng Hou
- Bioengineering College of Chongqing University, Chongqing, China; Key Laboratory of Biorheological Science and Technology of Ministry of Education, Chongqing University, Chongqing, China.
| |
Collapse
|
12
|
Oh J, Kuenze C, Signorile JF, Andersen MS, Letter M, Best TM, Ripic Z, Emerson C, Eltoukhy M. Estimation of ground reaction forces during stair climbing in patients with ACL reconstruction using a depth sensor-driven musculoskeletal model. Gait Posture 2021; 84:232-237. [PMID: 33383533 DOI: 10.1016/j.gaitpost.2020.12.025] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/14/2020] [Revised: 12/16/2020] [Accepted: 12/21/2020] [Indexed: 02/02/2023]
Abstract
BACKGROUND Although stair ambulation should be included in the rehabilitation of the long-term effects of ACL injury on knee function, the assessment of kinetic parameter in the situation where stair gait can only be established using costly and cumbersome force platforms via conventional inverse dynamic analysis. Therefore, there is a need to develop a practical laboratory setup as an assessment tool of the stair gait abnormalities in lower extremity that arise from an ACL deficiency. RESEARCH QUESTION Can the use of a single depth sensor-driven full-body musculoskeletal gait model be considered an accurate assessment tool of the ground reaction forces (GRFs) during stair climbing for patients following ACL reconstruction (ACLR) surgery? METHODS A total of 15 patients who underwent ACLR participated in this study. GRFs data during stair climbing was collected using a custom-built 3-step staircase with two embedded force platforms. A single depth sensor, commercially available and cost effective, was used to obtain participants' depth map information to extract the full-body skeleton information. The AnyBody TM GaitFullBody model was utilized to estimate GRFs attained by 25 artificial muscle-like actuators placed under each foot. Mean differences between the measured and estimated GRFs were compared using paired samples t-tests. The ensemble curves of the GRFs were compared between both approaches during stance phase of the gait cycle. RESULTS The findings of this study showed that the estimation of the GRFs produced during staircase gait using a depth sensor-driven musculoskeletal model can produce acceptable results when compared to the traditional inverse dynamics modelling approach as an alternative tool in clinical settings for individuals who had undergone ACLR. SIGNIFICANCE The introduced approach of full-body musculoskeletal modelling driven by a single depth sensor has the potential to be a cost-effective stair gait analysis tool for patients with ACL injury.
Collapse
Affiliation(s)
- Jeonghoon Oh
- Department of Kinesiology and Sport Sciences, School of Education & Human Development, University of Miami, Coral Gables, FL, 33143, USA
| | - Christopher Kuenze
- Department of Kinesiology, School of Education, Michigan State University, East Lansing, MI, 48824, USA
| | - Joseph F Signorile
- Department of Kinesiology and Sport Sciences, School of Education & Human Development, University of Miami, Coral Gables, FL, 33143, USA; Center on Aging, Miller School of Medicine, University of Miami, Coral Gables, FL, 33146, USA
| | - Michael S Andersen
- Department of Materials and Production, Aalborg University, Fibigerstraede 16, 9220, Aalborg East, Denmark
| | - Michael Letter
- Orthopedic Sports Medicine, Miller School of Medicine, University of Miami, Coral Gables, FL, 33146, USA
| | - Thomas M Best
- Orthopedic Sports Medicine, Miller School of Medicine, University of Miami, Coral Gables, FL, 33146, USA
| | - Zachary Ripic
- Department of Kinesiology and Sport Sciences, School of Education & Human Development, University of Miami, Coral Gables, FL, 33143, USA
| | - Christopher Emerson
- Orthopedic Sports Medicine, Miller School of Medicine, University of Miami, Coral Gables, FL, 33146, USA
| | - Moataz Eltoukhy
- Department of Kinesiology and Sport Sciences, School of Education & Human Development, University of Miami, Coral Gables, FL, 33143, USA.
| |
Collapse
|
13
|
Takeda I, Yamada A, Onodera H. Artificial Intelligence-Assisted motion capture for medical applications: a comparative study between markerless and passive marker motion capture. Comput Methods Biomech Biomed Engin 2020; 24:864-873. [PMID: 33290107 DOI: 10.1080/10255842.2020.1856372] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/22/2022]
Abstract
We aimed to determine whether artificial intelligence (AI)-assisted markerless motion capture software is useful in the clinical medicine and rehabilitation fields. Currently, it is unclear whether the AI-assisted markerless method can be applied to individuals with lower limb dysfunction, such as those using an ankle foot orthosis or a crutch. However, as many patients with lower limb paralysis and foot orthosis users lose metatarsophalangeal (MP) joint flexion during the stance phase, it is necessary to estimate the accuracy of foot recognition under fixed MP joint motion. The hip, knee, and ankle joint angles during treadmill walking were determined using OpenPose (a markerless method) and the conventional passive marker motion capture method; the results from both methods were compared. We also examined whether an ankle foot orthosis and a crutch could influence the recognition ability of OpenPose. The hip and knee joint data obtained by the passive marker method (MAC3D), OpenPose, and manual video analysis using Kinovea software showed significant correlation. Compared with the ankle joint data obtained by OpenPose and Kinovea, which were strongly correlated, those obtained by MAC3D presented a weaker correlation. OpenPose can be an adequate substitute for conventional passive marker motion capture for both normal gait and abnormal gait with an orthosis or a crutch. Furthermore, OpenPose is applicable to patients with impaired MP joint motion. The use of OpenPose can reduce the complexity and cost associated with conventional passive marker motion capture without compromising recognition accuracy.
Collapse
Affiliation(s)
- Iwori Takeda
- Department of Mechanical Systems Engineering, School of Engineering, The University of Tokyo, Tokyo, Japan
| | - Atsushi Yamada
- Department of Mechanical Systems Engineering, School of Engineering, The University of Tokyo, Tokyo, Japan
| | - Hiroshi Onodera
- Department of Mechanical Systems Engineering, School of Engineering, The University of Tokyo, Tokyo, Japan
| |
Collapse
|
14
|
Dajime PF, Smith H, Zhang Y. Automated classification of movement quality using the Microsoft Kinect V2 sensor. Comput Biol Med 2020; 125:104021. [PMID: 33011646 DOI: 10.1016/j.compbiomed.2020.104021] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/10/2020] [Revised: 09/22/2020] [Accepted: 09/23/2020] [Indexed: 11/26/2022]
Abstract
Practitioners commonly perform movement quality assessment through qualitative assessment protocols, which can be time-intensive and prone to inter-rater measurement bias. The advent of portable and inexpensive marker-less motion capture systems can improve assessment through objective joint kinematic analysis. The current study aimed to evaluate various machine learning models that used kinematic features from Kinect position data to classify a performer's Movement Competency Screen (MCS) score. A Kinect V2 sensor collected position data from 31 physically active males as they performed bilateral squat, forward lunge, and single-leg squat; and the movement quality was rated according to the MCS criteria. Features were extracted and selected from domain knowledge-based kinematic variables as model input. Multiclass logistic regression (MLR) was then performed to translate joint kinematics into MCS score. Performance indicators were calculated after a 10-fold cross validation of each model developed from Kinect-based kinematic variables. The analyses revealed that the models' sensitivity, specificity, and accuracy ranged from 0.66 to 0.89, 0.58 to 0.86, and 0.74 to 0.85, respectively. In conclusion, the Kinect-based automated movement quality assessment is a suitable, novel, and practical approach to movement quality assessment.
Collapse
Affiliation(s)
| | - Heather Smith
- Department of Exercise Sciences, University of Auckland, New Zealand
| | - Yanxin Zhang
- Department of Exercise Sciences, University of Auckland, New Zealand.
| |
Collapse
|
15
|
Zhu Y, Lu W, Wang Y, Yang J, Gan W. Extraction and selection of gait recognition features using skeleton point detection and improved fuzzy decision. Med Eng Phys 2020; 84:161-168. [PMID: 32977914 DOI: 10.1016/j.medengphy.2020.08.007] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/12/2019] [Revised: 08/20/2020] [Accepted: 08/20/2020] [Indexed: 01/29/2023]
Abstract
It is of great importance to effectively measure gait features and recognize the signature gait patterns for gait rehabilitation. In this work, we used a skeleton point detection to extract gait features and proposed an improved fuzzy decision to select the most significant features for classifying gait patterns. Thirteen gait recognition features were extracted from the obtained skeleton points data. Taking the extracted features as an input, our improved fuzzy similarity priority decision method has obtained important sequences of all features based on the relatively important scores. Then, the ranked features were delivered in different classifiers by a sequential forward selection strategy to select the optimal feature subset. There were significant differences between groups in each of the thirteen gait recognition features (p < 0.005), indicating that all extracted features are potential influence factors for classifying gait patterns. We also found that the highest classification accuracy of 100% for gait feature subsets included the stride frequency, maximum flexion angle of knee, and toe-out angle, on the all classifiers. The results suggest that the proposed approaches are very useful in searching for the optimal feature subset in present dataset.
Collapse
Affiliation(s)
- Yean Zhu
- Bioengineering College, Chongqing University, Chongqing, China; School of Transportation and Logistics, East China Jiaotong University, Nanchang, China
| | - Wei Lu
- Department of Rehabilitation Medicine, Jiangxi Provincial People's Hospital, Nanchang, China.
| | - Yong Wang
- School of Mechanical Engineering, HeFei University of Technology, HeFei, China
| | - Jingjing Yang
- School of Public Health and Management, Chongqing Medical University, Chongqing, China.
| | - Weihua Gan
- School of Transportation and Logistics, East China Jiaotong University, Nanchang, China
| |
Collapse
|
16
|
Comparison of predicted kinetic variables between Parkinson's disease patients and healthy age-matched control using a depth sensor-driven full-body musculoskeletal model. Gait Posture 2020; 76:151-156. [PMID: 31862662 DOI: 10.1016/j.gaitpost.2019.11.011] [Citation(s) in RCA: 14] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/04/2019] [Revised: 10/31/2019] [Accepted: 11/22/2019] [Indexed: 02/02/2023]
Abstract
BACKGROUND Abnormalities in gait kinetics in patients with Parkinson's disease (PD) who have suffer from gait impairment have been noted using a conventional inverse dynamic analysis derived by marker-based motion capture system and force plate, which are typically mounted in the laboratory floor. Despite the high accuracy of this approach in tracking markers' trajectories and acquiring ground reaction forces (GRFs), its dependence on laboratory-mounted equipment restricts its potential use in wider variety of clinical applications. RESEARCH QUESTION Would a full-body musculoskeletal model driven by a single depth sensor data only produce comparable gait kinetic parameters, including GRFs and lower extremity joints moments, for elderly participants, both healthy and those diagnosed with PD? METHODS Nine patients diagnosed with PD and 11 healthy age-matched control participants performed three over-ground gait trials. Full-body kinematic data were collected using a depth sensor and a musculoskeletal model have been constructed using AnyBody musculoskeletal modeling system to predict the three-dimensional GRFs and lower extremity joint moments. Predicted kinetic parameters for both PD and control groups were compared during the braking and propulsive phases of the gait cycle. In addition, ensemble curve analysis with 90% confidence intervals were constructed to compare between group differences across the stance phase of the gait cycle. RESULTS The findings of this study showed that the PD exhibited a significantly lower braking peak vertical GRF and propulsion peak horizontal GRF while no significant between-group differences were found in peak lower extremity joint moments. However, the PD showed significant alterations in lower extremity joint moments during the early and late phases of stance, which indicate a difference in ambulation strategy. SIGNIFICANCE The proposed method adopting full-body musculoskeletal model driven by a depth sensor data proves that it has the potential to be a portable and cost-effective gait analysis tool in the clinical setting.
Collapse
|
17
|
Çubukçu B, Yüzgeç U, Zileli R, Zileli A. Reliability and validity analyzes of Kinect V2 based measurement system for shoulder motions. Med Eng Phys 2019; 76:20-31. [PMID: 31882393 DOI: 10.1016/j.medengphy.2019.10.017] [Citation(s) in RCA: 21] [Impact Index Per Article: 4.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/08/2019] [Revised: 10/06/2019] [Accepted: 10/20/2019] [Indexed: 11/15/2022]
Abstract
Telerehabilitation systems provide some advantages against the classic rehabilitation methods. The ability of the shoulders depends on active motion range of them to do activities in daily life and to do sports. To evaluate the shoulder motions, range of motion (ROM) measurement is a basic method. Clinical goniometer and digital goniometer are the most commonly used measurement tools. However, these measurement tools have some deficiencies and difficulties. In this paper, we consider a Kinect One Sensor (Kinect V2) based measurement system for shoulder motions as an alternative method. The aim of this study is to examine the reliability and validity analyzes of the proposed shoulder measurement system. Three systems were used to evaluate validity of the Kinect V2 to measure shoulder motions: Kinect V2 based system, clinical goniometer and digital goniometer. One expert physical therapist measured shoulder abduction, flexion, external rotation, internal rotation and extension ROM values using a clinical goniometer and a digital goniometer in 40 healthy volunteers (22 males, 18 females, and 19-33 years old). All poses for each shoulder motion were captured with the Kinect V2 based system again and the ROM values were calculated. These procedures were carried out with all of the volunteer participants in three repetitions. In reliability for Kinect V2 based shoulder motion measurement system, we used the intraclass correlation coefficients (ICC), standard error of the measure (SEM), minimal detectable change (MDC). The validity test includes the 95% limits of agreement (LOA) and mean difference between the Kinect V2 based system and the both of the goniometer systems for measuring shoulder motions. The high ICC values show that the Kinect V2 based shoulder motion measurement system has very good intra-rater reliability for abduction, flexion, external rotation, internal rotation shoulder poses. For extension pose, it has good reliability result according to the ICC value. The validity analysis gives good results for all shoulder poses except internal rotation between Kinect V2 and clinical/digital goniometer. As a result, Kinect V2 based measurement system is a reliable and valid alternative telerehabilitation tool for shoulder motions.
Collapse
Affiliation(s)
- Burakhan Çubukçu
- Department of Computer Engineering, Bilecik Seyh Edebali University, 11210 Bilecik, Turkey
| | - Uğur Yüzgeç
- Department of Computer Engineering, Bilecik Seyh Edebali University, 11210 Bilecik, Turkey.
| | - Raif Zileli
- School of Health, Bilecik Seyh Edebali University, 11210 Bilecik, Turkey
| | - Ahu Zileli
- Department of Physical Medicine and Rehabilitation, Bilecik State Hospital, Bilecik, Turkey
| |
Collapse
|
18
|
The Validity and Reliability of a Kinect v2-Based Gait Analysis System for Children with Cerebral Palsy. SENSORS 2019; 19:s19071660. [PMID: 30959970 PMCID: PMC6479781 DOI: 10.3390/s19071660] [Citation(s) in RCA: 23] [Impact Index Per Article: 4.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/10/2019] [Revised: 03/29/2019] [Accepted: 04/04/2019] [Indexed: 11/23/2022]
Abstract
The aim of this study is to evaluate if Kinect is a valid and reliable clinical gait analysis tool for children with cerebral palsy (CP), and whether linear regression and long short-term memory (LSTM) recurrent neural network methods can improve its performance. A gait analysis was conducted on ten children with CP, on two occasions. Lower limb joint kinematics computed from the Kinect and a traditional marker-based Motion Analysis system were investigated by calculating the root mean square errors (RMSE), the coefficients of multiple correlation (CMC), and the intra-class correlation coefficients (ICC2,k). Results showed that the Kinect-based kinematics had an overall modest to poor correlation (CMC—less than 0.001 to 0.70) and an angle pattern similarity with Motion Analysis. After the calibration, RMSE on every degree of freedom decreased. The two calibration methods indicated similar levels of improvement in hip sagittal (CMC—0.81 ± 0.10 vs. 0.75 ± 0.22)/frontal (CMC—0.41 ± 0.35 vs. 0.42 ± 0.37) and knee sagittal kinematics (CMC—0.85±0.07 vs. 0.87 ± 0.12). The hip sagittal (CMC—0.97±0.05) and knee sagittal (CMC—0.88 ± 0.12) angle patterns showed a very good agreement over two days. Modest to excellent reliability (ICC2,k—0.45 to 0.93) for most parameters renders it feasible for observing ongoing changes in gait kinematics.
Collapse
|