1
|
Lee P, Chen TB, Lin HY, Yeh LR, Liu CH, Chen YL. Integrating OpenPose and SVM for Quantitative Postural Analysis in Young Adults: A Temporal-Spatial Approach. Bioengineering (Basel) 2024; 11:548. [PMID: 38927784 PMCID: PMC11200693 DOI: 10.3390/bioengineering11060548] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2024] [Revised: 05/12/2024] [Accepted: 05/25/2024] [Indexed: 06/28/2024] Open
Abstract
Noninvasive tracking devices are widely used to monitor real-time posture. Yet significant potential exists to enhance postural control quantification through walking videos. This study advances computational science by integrating OpenPose with a Support Vector Machine (SVM) to perform highly accurate and robust postural analysis, marking a substantial improvement over traditional methods which often rely on invasive sensors. Utilizing OpenPose-based deep learning, we generated Dynamic Joint Nodes Plots (DJNP) and iso-block postural identity images for 35 young adults in controlled walking experiments. Through Temporal and Spatial Regression (TSR) models, key features were extracted for SVM classification, enabling the distinction between various walking behaviors. This approach resulted in an overall accuracy of 0.990 and a Kappa index of 0.985. Cutting points for the ratio of top angles (TAR) and the ratio of bottom angles (BAR) effectively differentiated between left and right skews with AUC values of 0.772 and 0.775, respectively. These results demonstrate the efficacy of integrating OpenPose with SVM, providing more precise, real-time analysis without invasive sensors. Future work will focus on expanding this method to a broader demographic, including individuals with gait abnormalities, to validate its effectiveness across diverse clinical conditions. Furthermore, we plan to explore the integration of alternative machine learning models, such as deep neural networks, enhancing the system's robustness and adaptability for complex dynamic environments. This research opens new avenues for clinical applications, particularly in rehabilitation and sports science, promising to revolutionize noninvasive postural analysis.
Collapse
Affiliation(s)
- Posen Lee
- Department of Occupational Therapy, College of Medicine, I-Shou University, Kaohsiung 82445, Taiwan;
| | - Tai-Been Chen
- Department of Radiological Technology, Faculty of Medical Technology, Teikyo University, Tokyo 173-8605, Japan;
| | - Hung-Yu Lin
- Department of Occupational Therapy, College of Medical and Health Science, Asia University, Taichung 41354, Taiwan;
| | - Li-Ren Yeh
- Department of Anesthesiology, E-DA Cancer Hospital, I-Shou University, Kaohsiung 82445, Taiwan;
| | - Chin-Hsuan Liu
- Department of Occupational Therapy, College of Medicine, I-Shou University, Kaohsiung 82445, Taiwan;
| | - Yen-Lin Chen
- Department of Computer Science and Information Engineering, College of Electrical Engineering and Computer Science, National Taipei University of Technology, Taipei 10608, Taiwan;
| |
Collapse
|
2
|
Arrowsmith C, Burns D, Mak T, Hardisty M, Whyne C. Physiotherapy Exercise Classification with Single-Camera Pose Detection and Machine Learning. SENSORS (BASEL, SWITZERLAND) 2022; 23:363. [PMID: 36616961 PMCID: PMC9824820 DOI: 10.3390/s23010363] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/03/2022] [Revised: 12/15/2022] [Accepted: 12/20/2022] [Indexed: 06/17/2023]
Abstract
Access to healthcare, including physiotherapy, is increasingly occurring through virtual formats. At-home adherence to physical therapy programs is often poor and few tools exist to objectively measure participation. The aim of this study was to develop and evaluate the potential for performing automatic, unsupervised video-based monitoring of at-home low-back and shoulder physiotherapy exercises using a mobile phone camera. Joint locations were extracted from the videos of healthy subjects performing low-back and shoulder physiotherapy exercises using an open source pose detection framework. A convolutional neural network was trained to classify physiotherapy exercises based on the segments of keypoint time series data. The model's performance as a function of input keypoint combinations was studied in addition to its robustness to variation in the camera angle. The CNN model achieved optimal performance using a total of 12 pose estimation landmarks from the upper and lower body (low-back exercise classification: 0.995 ± 0.009; shoulder exercise classification: 0.963 ± 0.020). Training the CNN on a variety of angles was found to be effective in making the model robust to variations in video filming angle. This study demonstrates the feasibility of using a smartphone camera and a supervised machine learning model to effectively classify at-home physiotherapy participation and could provide a low-cost, scalable method for tracking adherence to physical therapy exercise programs in a variety of settings.
Collapse
Affiliation(s)
- Colin Arrowsmith
- Orthopaedic Biomechanics Lab, Holland Bone and Joint Program, Sunnybrook Research Institute, Toronto, ON M4N 3M5, Canada
- Halterix Corporation, Toronto, ON M5E 1L4, Canada
| | - David Burns
- Orthopaedic Biomechanics Lab, Holland Bone and Joint Program, Sunnybrook Research Institute, Toronto, ON M4N 3M5, Canada
- Halterix Corporation, Toronto, ON M5E 1L4, Canada
- Division of Orthopaedic Surgery, University of Toronto, Toronto, ON M5T 1P5, Canada
| | - Thomas Mak
- Halterix Corporation, Toronto, ON M5E 1L4, Canada
| | - Michael Hardisty
- Orthopaedic Biomechanics Lab, Holland Bone and Joint Program, Sunnybrook Research Institute, Toronto, ON M4N 3M5, Canada
- Division of Orthopaedic Surgery, University of Toronto, Toronto, ON M5T 1P5, Canada
| | - Cari Whyne
- Orthopaedic Biomechanics Lab, Holland Bone and Joint Program, Sunnybrook Research Institute, Toronto, ON M4N 3M5, Canada
- Division of Orthopaedic Surgery, University of Toronto, Toronto, ON M5T 1P5, Canada
- Institute of Biomedical Engineering, University of Toronto, Toronto, ON M5S 3G9, Canada
| |
Collapse
|
3
|
Lee P, Chen TB, Liu CH, Wang CY, Huang GH, Lu NH. Identifying the Posture of Young Adults in Walking Videos by Using a Fusion Artificial Intelligent Method. BIOSENSORS 2022; 12:bios12050295. [PMID: 35624595 PMCID: PMC9139042 DOI: 10.3390/bios12050295] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/09/2022] [Revised: 04/29/2022] [Accepted: 05/02/2022] [Indexed: 11/23/2022]
Abstract
Many neurological and musculoskeletal disorders are associated with problems related to postural movement. Noninvasive tracking devices are used to record, analyze, measure, and detect the postural control of the body, which may indicate health problems in real time. A total of 35 young adults without any health problems were recruited for this study to participate in a walking experiment. An iso-block postural identity method was used to quantitatively analyze posture control and walking behavior. The participants who exhibited straightforward walking and skewed walking were defined as the control and experimental groups, respectively. Fusion deep learning was applied to generate dynamic joint node plots by using OpenPose-based methods, and skewness was qualitatively analyzed using convolutional neural networks. The maximum specificity and sensitivity achieved using a combination of ResNet101 and the naïve Bayes classifier were 0.84 and 0.87, respectively. The proposed approach successfully combines cell phone camera recordings, cloud storage, and fusion deep learning for posture estimation and classification.
Collapse
Affiliation(s)
- Posen Lee
- Department of Occupation Therapy, I-Shou University, No. 8, Yida Road, Jiaosu Village, Yanchao District, Kaohsiung 82445, Taiwan;
| | - Tai-Been Chen
- Department of Medical Imaging and Radiological Science, I-Shou University, No. 8, Yida Road, Jiaosu Village Yanchao District, Kaohsiung 82445, Taiwan; (T.-B.C.); (C.-Y.W.); (N.-H.L.)
- Institute of Statistics, National Yang Ming Chiao Tung University, No. 1001, University Road, Hsinchu 30010, Taiwan;
| | - Chin-Hsuan Liu
- Department of Occupation Therapy, I-Shou University, No. 8, Yida Road, Jiaosu Village, Yanchao District, Kaohsiung 82445, Taiwan;
- Department of Occupational Therapy, Kaohsiung Municipal Kai-Syuan Psychiatric Hospital, No. 130, Kaisyuan 2nd Road, Lingya District, Kaohsiung 80276, Taiwan
- Correspondence: ; Tel.: +886-7-6151100 (ext. 7516)
| | - Chi-Yuan Wang
- Department of Medical Imaging and Radiological Science, I-Shou University, No. 8, Yida Road, Jiaosu Village Yanchao District, Kaohsiung 82445, Taiwan; (T.-B.C.); (C.-Y.W.); (N.-H.L.)
| | - Guan-Hua Huang
- Institute of Statistics, National Yang Ming Chiao Tung University, No. 1001, University Road, Hsinchu 30010, Taiwan;
| | - Nan-Han Lu
- Department of Medical Imaging and Radiological Science, I-Shou University, No. 8, Yida Road, Jiaosu Village Yanchao District, Kaohsiung 82445, Taiwan; (T.-B.C.); (C.-Y.W.); (N.-H.L.)
- Department of Pharmacy, Tajen University, No. 20, Weixin Road, Yanpu Township, Pingtung County 90741, Taiwan
- Department of Radiology, E-DA Hospital, I-Shou University, No. 1, Yida Road, Jiaosu Village, Yanchao District, Kaohsiung City 82445, Taiwan
| |
Collapse
|