1
|
Sun H, Chen Y. A Rapid Response System for Elderly Safety Monitoring Using Progressive Hierarchical Action Recognition. IEEE Trans Neural Syst Rehabil Eng 2024; 32:2134-2142. [PMID: 38833396 DOI: 10.1109/tnsre.2024.3409197] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/06/2024]
Abstract
The global trend of population aging presents an urgent challenge in ensuring the safety and well-being of elderly individuals, especially those living alone due to various circumstances. A promising approach to this challenge involves leveraging Human Action Recognition (HAR) by integrating data from multiple sensors. However, the field of HAR has struggled to strike a balance between accuracy and response time. While technological advancements have improved recognition accuracy, complex algorithms often come at the expense of response time. To address this issue, we introduce an innovative asynchronous detection method called Rapid Response Elderly Safety Monitoring (RESAM), which relies on progressive hierarchical action recognition and multi-sensor data fusion. Through initial analysis of inertial sensor data using Kernel Principal Component Analysis (KPCA) and multi-class classifiers, we efficiently reduce processing time and lower the false-negative rate (FNR). The inertial sensor identification serves as a pre-filter, enabling the identification of filtered abnormal signals. Decision-level data fusion is then executed, incorporating skeleton image analysis based on ResNet and the inertial sensor data from the initial step. This integration enables the accurate differentiation between normal and abnormal behaviors. The RESAM method achieves an impressive 97.4% accuracy on the UTD-MHAD database with a minimal delay of 1.22 seconds. On our internally collected database, the RESAM system attains an accuracy of 99%, ranking among the most accurate state-of-the-art methods available. These results underscore the practicality and effectiveness of our approach in meeting the critical demand for swift and precise responses in healthcare scenarios.
Collapse
|
2
|
Yun SH, Kim HJ, Ryu JK, Kim SC. Fine-Grained Motion Recognition in At-Home Fitness Monitoring with Smartwatch: A Comparative Analysis of Explainable Deep Neural Networks. Healthcare (Basel) 2023; 11:healthcare11070940. [PMID: 37046868 PMCID: PMC10094383 DOI: 10.3390/healthcare11070940] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/15/2023] [Revised: 03/13/2023] [Accepted: 03/21/2023] [Indexed: 04/14/2023] Open
Abstract
The squat is a multi-joint exercise widely used for everyday at-home fitness. Focusing on the fine-grained classification of squat motions, we propose a smartwatch-based wearable system that can recognize subtle motion differences. For data collection, 52 participants were asked to perform one correct squat and five incorrect squats with three different arm postures (straight arm, crossed arm, and hands on waist). We utilized deep neural network-based models and adopted a conventional machine learning method (random forest) as a baseline. Experimental results revealed that the bidirectional GRU/LSTMs with an attention mechanism and the arm posture of hands on waist achieved the best test accuracy (F1-score) of 0.854 (0.856). High-dimensional embeddings in the latent space learned by attention-based models exhibit more clustered distributions than those by other DNN models, indicating that attention-based models learned features from the complex multivariate time-series motion signals more efficiently. To understand the underlying decision-making process of the machine-learning system, we analyzed the result of attention-based RNN models. The bidirectional GRU/LSTMs show a consistent pattern of attention for defined squat classes, but these models weigh the attention to the different kinematic events of the squat motion (e.g., descending and ascending). However, there was no significant difference found in classification performance.
Collapse
Affiliation(s)
- Seok-Ho Yun
- Department of Physical Education, Graduate School, Dongguk University, Seoul 04620, Republic of Korea
| | - Hyeon-Joo Kim
- Machine Learning Systems Lab., College of Sports Science, Sungkyunkwan University, Suwon 16419, Republic of Korea
| | - Jeh-Kwang Ryu
- Department of Physical Education, Graduate School, Dongguk University, Seoul 04620, Republic of Korea
| | - Seung-Chan Kim
- Machine Learning Systems Lab., College of Sports Science, Sungkyunkwan University, Suwon 16419, Republic of Korea
| |
Collapse
|
3
|
Yang J, Duan J, Li T, Hu C, Liang J, Shi T. Tool Wear Monitoring in Milling Based on Fine-Grained Image Classification of Machined Surface Images. SENSORS (BASEL, SWITZERLAND) 2022; 22:s22218416. [PMID: 36366114 PMCID: PMC9658698 DOI: 10.3390/s22218416] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/09/2022] [Revised: 10/25/2022] [Accepted: 10/31/2022] [Indexed: 05/14/2023]
Abstract
Cutting tool wear state assessment during the manufacturing process is extremely significant. The primary purpose of this study is to monitor tool wear to ensure timely tool change and avoid excessive tool wear or sudden tool breakage, which causes workpiece waste and could even damage the machine. Therefore, an intelligent system, that is efficient and precise, needs to be designed for addressing these problems. In our study, an end-to-end improved fine-grained image classification method is employed for workpiece surface-based tool wear monitoring, which is named efficient channel attention destruction and construction learning (ECADCL). The proposed method uses a feature extraction module to extract features from the input image and its corrupted images, and adversarial learning is used to avoid learning noise from corrupted images while extracting semantic features by reconstructing the corrupted images. Finally, a decision module predicts the label based on the learned features. Moreover, the feature extraction module combines a local cross-channel interaction attention mechanism without dimensionality reduction to characterize representative information. A milling dataset is conducted based on the machined surface images for monitoring tool wear conditions. The experimental results indicated that the proposed system can effectively assess the wear state of the tool.
Collapse
|
4
|
Kim HJ, Kim H, Park J, Oh B, Kim SC. Recognition of Gait Patterns in Older Adults Using Wearable Smartwatch Devices: Observational Study. J Med Internet Res 2022; 24:e39190. [PMID: 36001374 PMCID: PMC9453581 DOI: 10.2196/39190] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/03/2022] [Revised: 07/23/2022] [Accepted: 08/11/2022] [Indexed: 11/13/2022] Open
Affiliation(s)
- Hyeon-Joo Kim
- Machine Learning Systems Lab, College of Sports Science, Sungkyunkwan University, Suwon, Republic of Korea
| | - Hyejoo Kim
- Machine Learning Systems Lab, College of Sports Science, Sungkyunkwan University, Suwon, Republic of Korea
| | - Jinyoon Park
- Machine Learning Systems Lab, College of Sports Science, Sungkyunkwan University, Suwon, Republic of Korea
| | - Bumjo Oh
- Department of Family Medicine, Seoul Metropolitan Government - Seoul National University Boramae Medical Center, Seoul, Republic of Korea
| | - Seung-Chan Kim
- Machine Learning Systems Lab, College of Sports Science, Sungkyunkwan University, Suwon, Republic of Korea
| |
Collapse
|
5
|
Consumer Group Identification Algorithm for Ice and Snow Sports. COMPUTATIONAL INTELLIGENCE AND NEUROSCIENCE 2022; 2022:2174910. [PMID: 35814582 PMCID: PMC9259270 DOI: 10.1155/2022/2174910] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 02/18/2022] [Accepted: 05/25/2022] [Indexed: 11/17/2022]
Abstract
As an important part of the modern sports industry system, the quality and level of its development are related to whether China's sports industry can successfully become a pillar industry of the national economy. Therefore, the development of the ice and snow sports industry is to promote the expansion of China's sports industry scale high quality development of the national economy and an important way to build sports power. Participative sports consumption is the most important part of sports consumption and the development of the sports industry. The sports industry separated from participative sports consumption is water without source and tree without roots, while participative sports consumption demand is the power source of participative sports consumption. At present, there is no systematic and complete research on participation sports consumption demand. In order to understand the causes and demand state of residents' participation sports consumption demand and provide entry points for enterprises to formulate marketing strategies, this study constructs an organic system with participation sports service products as consumption objects, centering on the demanding state of participation sports consumers. In the system, on the theory of supply and demand, under the guidance of consumption economics theory, adhere to the combination of theoretical research and empirical analysis, the combination of macroplanning and microdesign, the combination of qualitative analysis and quantitative analysis, through the empirical investigation and receipt collection of residents' participation sports consumption demand, the use of systematic analysis, literature method, and survey method, through mathematical analysis, and other research methods, the paper explores the main causes and demand conditions of residents' participation sports consumption demand in different consumption states and excavates the main causes and demand conditions of participating sports consumption demand in different consumption states under different sports levels.
Collapse
|
6
|
Joo H, Kim H, Ryu JK, Ryu S, Lee KM, Kim SC. Estimation of Fine-Grained Foot Strike Patterns with Wearable Smartwatch Devices. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2022; 19:ijerph19031279. [PMID: 35162308 PMCID: PMC8835219 DOI: 10.3390/ijerph19031279] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 12/17/2021] [Revised: 01/19/2022] [Accepted: 01/21/2022] [Indexed: 12/14/2022]
Abstract
People who exercise may benefit or be injured depending on their foot striking (FS) style. In this study, we propose an intelligent system that can recognize subtle differences in FS patterns while walking and running using measurements from a wearable smartwatch device. Although such patterns could be directly measured utilizing pressure distribution of feet while striking on the ground, we instead focused on analyzing hand movements by assuming that striking patterns consequently affect temporal movements of the whole body. The advantage of the proposed approach is that FS patterns can be estimated in a portable and less invasive manner. To this end, first, we developed a wearable system for measuring inertial movements of hands and then conducted an experiment where participants were asked to walk and run while wearing a smartwatch. Second, we trained and tested the captured multivariate time series signals in supervised learning settings. The experimental results obtained demonstrated high and robust classification performances (weighted-average F1 score > 90%) when recent deep neural network models, such as 1D-CNN and GRUs, were employed. We conclude this study with a discussion of potential future work and applications that increase benefits while walking and running properly using the proposed approach.
Collapse
Affiliation(s)
- Hyeyeoun Joo
- Interdisciplinary Program in Cognitive Science, Seoul National University, Seoul 08826, Korea; (H.J.); (K.-M.L.)
| | - Hyejoo Kim
- Machine Learning Systems Laboratory, Department of Sports Science, Sungkyunkwan University, Suwon 16419, Korea;
| | - Jeh-Kwang Ryu
- Department of Physical Education, College of Education, Dongguk University, Seoul 04620, Korea;
| | - Semin Ryu
- Intelligent Robotics Laboratory, School of Artificial Intelligence Convergence, Hallym University, Chuncheon 24252, Korea;
| | - Kyoung-Min Lee
- Interdisciplinary Program in Cognitive Science, Seoul National University, Seoul 08826, Korea; (H.J.); (K.-M.L.)
| | - Seung-Chan Kim
- Machine Learning Systems Laboratory, Department of Sports Science, Sungkyunkwan University, Suwon 16419, Korea;
- Correspondence: ; Tel.: +82-31-299-6918
| |
Collapse
|
7
|
Deep Learning and Kurtosis-Controlled, Entropy-Based Framework for Human Gait Recognition Using Video Sequences. ELECTRONICS 2022. [DOI: 10.3390/electronics11030334] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/27/2023]
Abstract
Gait is commonly defined as the movement pattern of the limbs over a hard substrate, and it serves as a source of identification information for various computer-vision and image-understanding techniques. A variety of parameters, such as human clothing, angle shift, walking style, occlusion, and so on, have a significant impact on gait-recognition systems, making the scene quite complex to handle. In this article, we propose a system that effectively handles problems associated with viewing angle shifts and walking styles in a real-time environment. The following steps are included in the proposed novel framework: (a) real-time video capture, (b) feature extraction using transfer learning on the ResNet101 deep model, and (c) feature selection using the proposed kurtosis-controlled entropy (KcE) approach, followed by a correlation-based feature fusion step. The most discriminant features are then classified using the most advanced machine learning classifiers. The simulation process is fed by the CASIA B dataset as well as a real-time captured dataset. On selected datasets, the accuracy is 95.26% and 96.60%, respectively. When compared to several known techniques, the results show that our proposed framework outperforms them all.
Collapse
|