1
|
Lo CM, Hung PH. Predictive stroke risk model with vision transformer-based Doppler features. Med Phys 2024; 51:126-138. [PMID: 38043124 DOI: 10.1002/mp.16861] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/08/2023] [Revised: 11/17/2023] [Accepted: 11/17/2023] [Indexed: 12/05/2023] Open
Abstract
BACKGROUND Acute stroke is the leading cause of death and disability globally, with an estimated 16 million cases each year. The progression of carotid stenosis reduces blood flow to the intracranial vasculature, causing stroke. Early recognition of ischemic stroke is crucial for disease treatment and management. PURPOSE A computer-aided diagnosis (CAD) system was proposed in this study to rapidly evaluate ischemic stroke in carotid color Doppler (CCD). METHODS Based on the ground truth from the clinical examination report, the vision transformer (ViT) features extracted from all CCD images (513 stroke and 458 normal images) were combined in machine learning classifiers to generate the likelihood of ischemic stroke for each image. The pretrained weights from ImageNet reduced the time-consuming training process. The accuracy, sensitivity, specificity, and area under the receiver operating characteristic curve were calculated to evaluate the stroke prediction model. The chi-square test, DeLong test, and Bonferroni correction for multiple comparisons were applied to deal with the type-I error. Only p values equal to or less than 0.00125 were considered to be statistically significant. RESULTS The proposed CAD system achieved an accuracy of 89%, a sensitivity of 94%, a specificity of 84%, and an area under the receiver operating characteristic curve of 0.95, outperforming the convolutional neural networks AlexNet (82%, p < 0.001), Inception-v3 (78%, p < 0.001), ResNet101 (84%, p < 0.001), and DenseNet201 (85%, p < 0.01). The computational time in model training was only 30 s, which would be efficient and practical in clinical use. CONCLUSIONS The experiment shows the promising use of CCD images in stroke estimation. Using the pretrained ViT architecture, the image features can be automatically and efficiently generated without human intervention. The proposed CAD system provides a rapid and reliable suggestion for diagnosing ischemic stroke.
Collapse
Affiliation(s)
- Chung-Ming Lo
- Graduate Institute of Library, Information and Archival Studies, National Chengchi University, Taipei, Taiwan
| | - Peng-Hsiang Hung
- Department of Radiology, Mackay Memorial Hospital, Taipei, Taiwan
| |
Collapse
|
2
|
Weng X, Mei C, Gao F, Wu X, Zhang Q, Liu G. A gait stability evaluation method based on wearable acceleration sensors. MATHEMATICAL BIOSCIENCES AND ENGINEERING : MBE 2023; 20:20002-20024. [PMID: 38052634 DOI: 10.3934/mbe.2023886] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/07/2023]
Abstract
In this study, an accurate tool is provided for the evaluation of the effect of joint motion effect on gait stability. This quantitative gait evaluation method relies exclusively on the analysis of data acquired using acceleration sensors. First, the acceleration signal of lower limb motion is collected dynamically in real-time through the acceleration sensor. Second, an algorithm based on improved dynamic time warping (DTW) is proposed and used to calculate the gait stability index of the lower limbs. Finally, the effects of different joint braces on gait stability are analyzed. The experimental results show that the joint brace at the ankle and the knee reduces the range of motions of both ankle and knee joints, and a certain impact is exerted on the gait stability. In comparison to the ankle joint brace, the knee joint brace inflicts increased disturbance on the gait stability. Compared to the joint motion of the braced side, which showed a large deviation, the joint motion of the unbraced side was more similar to that of the normal walking process. In this paper, the quantitative evaluation algorithm based on DTW makes the results more intuitive and has potential application value in the evaluation of lower limb dysfunction, clinical training and rehabilitation.
Collapse
Affiliation(s)
- Xuecheng Weng
- School of Artificial Intelligence, Hangzhou Dianzi University, Hangzhou 310018, China
| | - Chang Mei
- School of Artificial Intelligence, Hangzhou Dianzi University, Hangzhou 310018, China
| | - Farong Gao
- School of Artificial Intelligence, Hangzhou Dianzi University, Hangzhou 310018, China
| | - Xudong Wu
- Department of Orthopaedics, Zhoushan Hospital of Traditional Chinese Medicine, Zhoushan 316000, China
| | - Qizhong Zhang
- School of Artificial Intelligence, Hangzhou Dianzi University, Hangzhou 310018, China
| | - Guangyu Liu
- School of Artificial Intelligence, Hangzhou Dianzi University, Hangzhou 310018, China
| |
Collapse
|
3
|
Khaliluzzaman M, Uddin A, Deb K, Hasan MJ. Person Recognition Based on Deep Gait: A Survey. SENSORS (BASEL, SWITZERLAND) 2023; 23:4875. [PMID: 37430786 DOI: 10.3390/s23104875] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/18/2023] [Revised: 05/12/2023] [Accepted: 05/16/2023] [Indexed: 07/12/2023]
Abstract
Gait recognition, also known as walking pattern recognition, has expressed deep interest in the computer vision and biometrics community due to its potential to identify individuals from a distance. It has attracted increasing attention due to its potential applications and non-invasive nature. Since 2014, deep learning approaches have shown promising results in gait recognition by automatically extracting features. However, recognizing gait accurately is challenging due to the covariate factors, complexity and variability of environments, and human body representations. This paper provides a comprehensive overview of the advancements made in this field along with the challenges and limitations associated with deep learning methods. For that, it initially examines the various gait datasets used in the literature review and analyzes the performance of state-of-the-art techniques. After that, a taxonomy of deep learning methods is presented to characterize and organize the research landscape in this field. Furthermore, the taxonomy highlights the basic limitations of deep learning methods in the context of gait recognition. The paper is concluded by focusing on the present challenges and suggesting several research directions to improve the performance of gait recognition in the future.
Collapse
Affiliation(s)
- Md Khaliluzzaman
- Department of Computer Science and Engineering, Chittagong University of Engineering & Technology, Chattogram 4349, Bangladesh
- Department of Computer Science and Engineering, International Islamic University Chittagong, Chattogram 4318, Bangladesh
| | - Ashraf Uddin
- Department of Computer Science and Engineering, Chittagong University of Engineering & Technology, Chattogram 4349, Bangladesh
| | - Kaushik Deb
- Department of Computer Science and Engineering, Chittagong University of Engineering & Technology, Chattogram 4349, Bangladesh
| | - Md Junayed Hasan
- National Subsea Centre, Robert Gordon University, Aberdeen AB10 7AQ, UK
| |
Collapse
|
4
|
Mogan JN, Lee CP, Lim KM, Ali M, Alqahtani A. Gait-CNN-ViT: Multi-Model Gait Recognition with Convolutional Neural Networks and Vision Transformer. SENSORS (BASEL, SWITZERLAND) 2023; 23:3809. [PMID: 37112147 PMCID: PMC10143319 DOI: 10.3390/s23083809] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 02/10/2023] [Revised: 03/08/2023] [Accepted: 03/13/2023] [Indexed: 06/19/2023]
Abstract
Gait recognition, the task of identifying an individual based on their unique walking style, can be difficult because walking styles can be influenced by external factors such as clothing, viewing angle, and carrying conditions. To address these challenges, this paper proposes a multi-model gait recognition system that integrates Convolutional Neural Networks (CNNs) and Vision Transformer. The first step in the process is to obtain a gait energy image, which is achieved by applying an averaging technique to a gait cycle. The gait energy image is then fed into three different models, DenseNet-201, VGG-16, and a Vision Transformer. These models are pre-trained and fine-tuned to encode the salient gait features that are specific to an individual's walking style. Each model provides prediction scores for the classes based on the encoded features, and these scores are then summed and averaged to produce the final class label. The performance of this multi-model gait recognition system was evaluated on three datasets, CASIA-B, OU-ISIR dataset D, and OU-ISIR Large Population dataset. The experimental results showed substantial improvement compared to existing methods on all three datasets. The integration of CNNs and ViT allows the system to learn both the pre-defined and distinct features, providing a robust solution for gait recognition even under the influence of covariates.
Collapse
Affiliation(s)
- Jashila Nair Mogan
- Faculty of Information Science and Technology, Multimedia University, Melaka 75450, Malaysia
| | - Chin Poo Lee
- Faculty of Information Science and Technology, Multimedia University, Melaka 75450, Malaysia
| | - Kian Ming Lim
- Faculty of Information Science and Technology, Multimedia University, Melaka 75450, Malaysia
| | - Mohammed Ali
- Department of Computer Science, King Khalid University, Abha 61421, Saudi Arabia
| | - Ali Alqahtani
- Department of Computer Science, King Khalid University, Abha 61421, Saudi Arabia
- Center for Artificial Intelligence (CAI), King Khalid University, Abha 61421, Saudi Arabia
| |
Collapse
|
5
|
Lai DKH, Yu ZH, Leung TYN, Lim HJ, Tam AYC, So BPH, Mao YJ, Cheung DSK, Wong DWC, Cheung JCW. Vision Transformers (ViT) for Blanket-Penetrating Sleep Posture Recognition Using a Triple Ultra-Wideband (UWB) Radar System. SENSORS (BASEL, SWITZERLAND) 2023; 23:2475. [PMID: 36904678 PMCID: PMC10006965 DOI: 10.3390/s23052475] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 01/19/2023] [Revised: 02/16/2023] [Accepted: 02/20/2023] [Indexed: 06/18/2023]
Abstract
Sleep posture has a crucial impact on the incidence and severity of obstructive sleep apnea (OSA). Therefore, the surveillance and recognition of sleep postures could facilitate the assessment of OSA. The existing contact-based systems might interfere with sleeping, while camera-based systems introduce privacy concerns. Radar-based systems might overcome these challenges, especially when individuals are covered with blankets. The aim of this research is to develop a nonobstructive multiple ultra-wideband radar sleep posture recognition system based on machine learning models. We evaluated three single-radar configurations (top, side, and head), three dual-radar configurations (top + side, top + head, and side + head), and one tri-radar configuration (top + side + head), in addition to machine learning models, including CNN-based networks (ResNet50, DenseNet121, and EfficientNetV2) and vision transformer-based networks (traditional vision transformer and Swin Transformer V2). Thirty participants (n = 30) were invited to perform four recumbent postures (supine, left side-lying, right side-lying, and prone). Data from eighteen participants were randomly chosen for model training, another six participants' data (n = 6) for model validation, and the remaining six participants' data (n = 6) for model testing. The Swin Transformer with side and head radar configuration achieved the highest prediction accuracy (0.808). Future research may consider the application of the synthetic aperture radar technique.
Collapse
Affiliation(s)
- Derek Ka-Hei Lai
- Department of Biomedical Engineering, Faculty of Engineering, The Hong Kong Polytechnic University, Hong Kong 999077, China
| | - Zi-Han Yu
- School of Engineering Sciences, Huazhong University of Science and Technology, Wuhan 430074, China
| | - Tommy Yau-Nam Leung
- Department of Biomedical Engineering, Faculty of Engineering, The Hong Kong Polytechnic University, Hong Kong 999077, China
| | - Hyo-Jung Lim
- Department of Biomedical Engineering, Faculty of Engineering, The Hong Kong Polytechnic University, Hong Kong 999077, China
| | - Andy Yiu-Chau Tam
- Department of Biomedical Engineering, Faculty of Engineering, The Hong Kong Polytechnic University, Hong Kong 999077, China
| | - Bryan Pak-Hei So
- Department of Biomedical Engineering, Faculty of Engineering, The Hong Kong Polytechnic University, Hong Kong 999077, China
| | - Ye-Jiao Mao
- Department of Biomedical Engineering, Faculty of Engineering, The Hong Kong Polytechnic University, Hong Kong 999077, China
| | - Daphne Sze Ki Cheung
- School of Nursing, The Hong Kong Polytechnic University, Hong Kong 999077, China
| | - Duo Wai-Chi Wong
- Department of Biomedical Engineering, Faculty of Engineering, The Hong Kong Polytechnic University, Hong Kong 999077, China
| | - James Chung-Wai Cheung
- Department of Biomedical Engineering, Faculty of Engineering, The Hong Kong Polytechnic University, Hong Kong 999077, China
- Research Institute of Smart Ageing, The Hong Kong Polytechnic University, Hong Kong 999077, China
| |
Collapse
|
6
|
Derlatka M, Borowska M. Ensemble of Heterogeneous Base Classifiers for Human Gait Recognition. SENSORS (BASEL, SWITZERLAND) 2023; 23:508. [PMID: 36617105 PMCID: PMC9824449 DOI: 10.3390/s23010508] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/20/2022] [Revised: 12/23/2022] [Accepted: 12/26/2022] [Indexed: 06/17/2023]
Abstract
Human gait recognition is one of the most interesting issues within the subject of behavioral biometrics. The most significant problems connected with the practical application of biometric systems include their accuracy as well as the speed at which they operate, understood both as the time needed to recognize a particular person as well as the time necessary to create and train a biometric system. The present study made use of an ensemble of heterogeneous base classifiers to address these issues. A Heterogeneous ensemble is a group of classification models trained using various algorithms and combined to output an effective recognition A group of parameters identified on the basis of ground reaction forces was accepted as input signals. The proposed solution was tested on a sample of 322 people (5980 gait cycles). Results concerning the accuracy of recognition (meaning the Correct Classification Rate quality at 99.65%), as well as operation time (meaning the time of model construction at <12.5 min and the time needed to recognize a person at <0.1 s), should be considered as very good and exceed in quality other methods so far described in the literature.
Collapse
|