1
|
Fu Y, Zhang Y, Ye B, Babineau J, Zhao Y, Gao Z, Mihailidis A. Smartphone-Based Hand Function Assessment: Systematic Review. J Med Internet Res 2024; 26:e51564. [PMID: 39283676 PMCID: PMC11443181 DOI: 10.2196/51564] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/03/2023] [Revised: 03/05/2024] [Accepted: 07/24/2024] [Indexed: 10/04/2024] Open
Abstract
BACKGROUND Hand function assessment heavily relies on specific task scenarios, making it challenging to ensure validity and reliability. In addition, the wide range of assessment tools, limited and expensive data recording, and analysis systems further aggravate the issue. However, smartphones provide a promising opportunity to address these challenges. Thus, the built-in, high-efficiency sensors in smartphones can be used as effective tools for hand function assessment. OBJECTIVE This review aims to evaluate existing studies on hand function evaluation using smartphones. METHODS An information specialist searched 8 databases on June 8, 2023. The search criteria included two major concepts: (1) smartphone or mobile phone or mHealth and (2) hand function or function assessment. Searches were limited to human studies in the English language and excluded conference proceedings and trial register records. Two reviewers independently screened all studies, with a third reviewer involved in resolving discrepancies. The included studies were rated according to the Mixed Methods Appraisal Tool. One reviewer extracted data on publication, demographics, hand function types, sensors used for hand function assessment, and statistical or machine learning (ML) methods. Accuracy was checked by another reviewer. The data were synthesized and tabulated based on each of the research questions. RESULTS In total, 46 studies were included. Overall, 11 types of hand dysfunction-related problems were identified, such as Parkinson disease, wrist injury, stroke, and hand injury, and 6 types of hand dysfunctions were found, namely an abnormal range of motion, tremors, bradykinesia, the decline of fine motor skills, hypokinesia, and nonspecific dysfunction related to hand arthritis. Among all built-in smartphone sensors, the accelerometer was the most used, followed by the smartphone camera. Most studies used statistical methods for data processing, whereas ML algorithms were applied for disease detection, disease severity evaluation, disease prediction, and feature aggregation. CONCLUSIONS This systematic review highlights the potential of smartphone-based hand function assessment. The review suggests that a smartphone is a promising tool for hand function evaluation. ML is a conducive method to classify levels of hand dysfunction. Future research could (1) explore a gold standard for smartphone-based hand function assessment and (2) take advantage of smartphones' multiple built-in sensors to assess hand function comprehensively, focus on developing ML methods for processing collected smartphone data, and focus on real-time assessment during rehabilitation training. The limitations of the research are 2-fold. First, the nascent nature of smartphone-based hand function assessment led to limited relevant literature, affecting the evidence's completeness and comprehensiveness. This can hinder supporting viewpoints and drawing conclusions. Second, literature quality varies due to the exploratory nature of the topic, with potential inconsistencies and a lack of high-quality reference studies and meta-analyses.
Collapse
Affiliation(s)
- Yan Fu
- School of Mechanical Science and Engineering, Huazhong University of Science and Technology, Wuhan, China
| | - Yuxin Zhang
- School of Mechanical Science and Engineering, Huazhong University of Science and Technology, Wuhan, China
| | - Bing Ye
- KITE - Toronto Rehabilitation Institute, University Health Network, Toronto, ON, Canada
- Department of Occupational Science and Occupational Therapy, University of Toronto, Toronto, ON, Canada
| | - Jessica Babineau
- Library and Information Services, University Health Network, Toronto, ON, Canada
| | - Yan Zhao
- Department of Rehabilitation Medicine, Hubei Province Academy of Traditional Chinese Medicine Hubei Provincial Hospital of Traditional Chinese Medicine, Wuhan, China
| | - Zhengke Gao
- School of Mechanical Science and Engineering, Huazhong University of Science and Technology, Wuhan, China
| | - Alex Mihailidis
- KITE - Toronto Rehabilitation Institute, University Health Network, Toronto, ON, Canada
- Department of Occupational Science and Occupational Therapy, University of Toronto, Toronto, ON, Canada
| |
Collapse
|
2
|
Mobbs A, Kahn M, Williams G, Mentiplay BF, Pua YH, Clark RA. Machine learning for automating subjective clinical assessment of gait impairment in people with acquired brain injury - a comparison of an image extraction and classification system to expert scoring. J Neuroeng Rehabil 2024; 21:124. [PMID: 39039594 PMCID: PMC11264460 DOI: 10.1186/s12984-024-01406-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/07/2024] [Accepted: 06/14/2024] [Indexed: 07/24/2024] Open
Abstract
BACKGROUND Walking impairment is a common disability post acquired brain injury (ABI), with visually evident arm movement abnormality identified as negatively impacting a multitude of psychological factors. The International Classification of Functioning, Disability and Health (ICF) qualifiers scale has been used to subjectively assess arm movement abnormality, showing strong intra-rater and test-retest reliability, however, only moderate inter-rater reliability. This impacts clinical utility, limiting its use as a measurement tool. To both automate the analysis and overcome these errors, the primary aim of this study was to evaluate the ability of a novel two-level machine learning model to assess arm movement abnormality during walking in people with ABI. METHODS Frontal plane gait videos were used to train four networks with 50%, 75%, 90%, and 100% of participants (ABI: n = 42, healthy controls: n = 34) to automatically identify anatomical landmarks using DeepLabCut™ and calculate two-dimensional kinematic joint angles. Assessment scores from three experienced neurorehabilitation clinicians were used with these joint angles to train random forest networks with nested cross-validation to predict assessor scores for all videos. Agreement between unseen participant (i.e. test group participants that were not used to train the model) predictions and each individual assessor's scores were compared using quadratic weighted kappa. One sample t-tests (to determine over/underprediction against clinician ratings) and one-way ANOVA (to determine differences between networks) were applied to the four networks. RESULTS The machine learning predictions have similar agreement to experienced human assessors, with no statistically significant (p < 0.05) difference for any match contingency. There was no statistically significant difference between the predictions from the four networks (F = 0.119; p = 0.949). The four networks did however under-predict scores with small effect sizes (p range = 0.007 to 0.040; Cohen's d range = 0.156 to 0.217). CONCLUSIONS This study demonstrated that machine learning can perform similarly to experienced clinicians when subjectively assessing arm movement abnormality in people with ABI. The relatively small sample size may have resulted in under-prediction of some scores, albeit with small effect sizes. Studies with larger sample sizes that objectively and automatically assess dynamic movement in both local and telerehabilitation assessments, for example using smartphones and edge-based machine learning, to reduce measurement error and healthcare access inequality are needed.
Collapse
Affiliation(s)
- Ashleigh Mobbs
- School of Health, University of the Sunshine Coast, Sippy Downs, QLD, Australia
| | - Michelle Kahn
- Department of Physiotherapy, Epworth Healthcare, Richmond, VIC, Australia
| | - Gavin Williams
- Department of Physiotherapy, Epworth Healthcare, Richmond, VIC, Australia
- School of Health Sciences, University of Melbourne, Parkville, VIC, Australia
| | - Benjamin F Mentiplay
- School of Allied Health, Human Services and Sport, La Trobe University, Bundoora, VIC, Australia
| | - Yong-Hao Pua
- Department of Physiotherapy, Singapore General Hospital, Singapore, Singapore
- Duke-National University of Singapore Medical School, Singapore, Singapore
| | - Ross A Clark
- School of Health, University of the Sunshine Coast, Sippy Downs, QLD, Australia.
| |
Collapse
|
3
|
Haberfehlner H, Roth Z, Vanmechelen I, Buizer AI, Jeroen Vermeulen R, Koy A, Aerts JM, Hallez H, Monbaliu E. A Novel Video-Based Methodology for Automated Classification of Dystonia and Choreoathetosis in Dyskinetic Cerebral Palsy During a Lower Extremity Task. Neurorehabil Neural Repair 2024; 38:479-492. [PMID: 38842031 DOI: 10.1177/15459683241257522] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/07/2024]
Abstract
BACKGROUND Movement disorders in children and adolescents with dyskinetic cerebral palsy (CP) are commonly assessed from video recordings, however scoring is time-consuming and expert knowledge is required for an appropriate assessment. OBJECTIVE To explore a machine learning approach for automated classification of amplitude and duration of distal leg dystonia and choreoathetosis within short video sequences. METHODS Available videos of a heel-toe tapping task were preprocessed to optimize key point extraction using markerless motion analysis. Postprocessed key point data were passed to a time series classification ensemble algorithm to classify dystonia and choreoathetosis duration and amplitude classes (scores 0, 1, 2, 3, and 4), respectively. As ground truth clinical scoring of dystonia and choreoathetosis by the Dyskinesia Impairment Scale was used. Multiclass performance metrics as well as metrics for summarized scores: absence (score 0) and presence (score 1-4) were determined. RESULTS Thirty-three participants were included: 29 with dyskinetic CP and 4 typically developing, age 14 years:6 months ± 5 years:15 months. The multiclass accuracy results for dystonia were 77% for duration and 68% for amplitude; for choreoathetosis 30% for duration and 38% for amplitude. The metrics for score 0 versus score 1 to 4 revealed an accuracy of 81% for dystonia duration, 77% for dystonia amplitude, 53% for choreoathetosis duration and amplitude. CONCLUSIONS This methodology study yielded encouraging results in distinguishing between presence and absence of dystonia, but not for choreoathetosis. A larger dataset is required for models to accurately represent distinct classes/scores. This study presents a novel methodology of automated assessment of movement disorders solely from video data.
Collapse
Affiliation(s)
- Helga Haberfehlner
- Department of Rehabilitation Sciences, KU Leuven Bruges, Bruges, Belgium
- Department of Rehabilitation Sciences, KU Leuven, Leuven, Belgium
- Amsterdam UMC, Vrije Universiteit Amsterdam, Department of Rehabilitation Medicine, Amsterdam, The Netherlands
- Amsterdam Movement Sciences, Rehabilitation & Development, Amsterdam, The Netherlands
| | - Zachary Roth
- Department of Rehabilitation Sciences, KU Leuven Bruges, Bruges, Belgium
- Department of Rehabilitation Sciences, KU Leuven, Leuven, Belgium
| | - Inti Vanmechelen
- Department of Rehabilitation Sciences, KU Leuven Bruges, Bruges, Belgium
- Department of Rehabilitation Sciences, KU Leuven, Leuven, Belgium
| | - Annemieke I Buizer
- Amsterdam UMC, Vrije Universiteit Amsterdam, Department of Rehabilitation Medicine, Amsterdam, The Netherlands
- Amsterdam Movement Sciences, Rehabilitation & Development, Amsterdam, The Netherlands
- Amsterdam UMC, Emma Children's Hospital, Amsterdam, The Netherlands
| | | | - Anne Koy
- Department of Pediatrics, University of Cologne, Faculty of Medicine and University Hospital Cologne, Cologne, Germany
| | - Jean-Marie Aerts
- Department of Computer Science, Mechatronics Research Group (M-Group), KU Leuven Bruges, Distrinet, Bruges, Belgium
| | - Hans Hallez
- Department of Biosystems, Division of Animal and Human Health Engineering, Measure, Model and Manage Bioresponse (M3-BIORES), KU Leuven, Leuven, Belgium
| | - Elegast Monbaliu
- Department of Rehabilitation Sciences, KU Leuven Bruges, Bruges, Belgium
- Department of Rehabilitation Sciences, KU Leuven, Leuven, Belgium
| |
Collapse
|
4
|
Friedrich MU, Roenn AJ, Palmisano C, Alty J, Paschen S, Deuschl G, Ip CW, Volkmann J, Muthuraman M, Peach R, Reich MM. Validation and application of computer vision algorithms for video-based tremor analysis. NPJ Digit Med 2024; 7:165. [PMID: 38906946 PMCID: PMC11192937 DOI: 10.1038/s41746-024-01153-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/01/2023] [Accepted: 05/29/2024] [Indexed: 06/23/2024] Open
Abstract
Tremor is one of the most common neurological symptoms. Its clinical and neurobiological complexity necessitates novel approaches for granular phenotyping. Instrumented neurophysiological analyses have proven useful, but are highly resource-intensive and lack broad accessibility. In contrast, bedside scores are simple to administer, but lack the granularity to capture subtle but relevant tremor features. We utilise the open-source computer vision pose tracking algorithm Mediapipe to track hands in clinical video recordings and use the resulting time series to compute canonical tremor features. This approach is compared to marker-based 3D motion capture, wrist-worn accelerometry, clinical scoring and a second, specifically trained tremor-specific algorithm in two independent clinical cohorts. These cohorts consisted of 66 patients diagnosed with essential tremor, assessed in different task conditions and states of deep brain stimulation therapy. We find that Mediapipe-derived tremor metrics exhibit high convergent clinical validity to scores (Spearman's ρ = 0.55-0.86, p≤ .01) as well as an accuracy of up to 2.60 mm (95% CI [-3.13, 8.23]) and ≤0.21 Hz (95% CI [-0.05, 0.46]) for tremor amplitude and frequency measurements, matching gold-standard equipment. Mediapipe, but not the disease-specific algorithm, was capable of analysing videos involving complex configurational changes of the hands. Moreover, it enabled the extraction of tremor features with diagnostic and prognostic relevance, a dimension which conventional tremor scores were unable to provide. Collectively, this demonstrates that current computer vision algorithms can be transformed into an accurate and highly accessible tool for video-based tremor analysis, yielding comparable results to gold standard tremor recordings.
Collapse
Affiliation(s)
- Maximilian U Friedrich
- Center for Brain Circuit Therapeutics, Brigham and Women's Hospital, Boston, MA, USA.
- Harvard Medical School, Boston, MA, USA.
- Department of Neurology, University Hospital Wurzburg, Wuerzburg, Germany.
| | - Anna-Julia Roenn
- Department of Neurology, University Hospital Wurzburg, Wuerzburg, Germany
| | - Chiara Palmisano
- Department of Neurology, University Hospital Wurzburg, Wuerzburg, Germany
| | - Jane Alty
- Wicking Dementia Research and Education Centre, College of Health and Medicine, University of Tasmania, Hobart, Tasmania, Australia
| | | | | | - Chi Wang Ip
- Department of Neurology, University Hospital Wurzburg, Wuerzburg, Germany
| | - Jens Volkmann
- Department of Neurology, University Hospital Wurzburg, Wuerzburg, Germany
| | | | - Robert Peach
- Department of Neurology, University Hospital Wurzburg, Wuerzburg, Germany
- Department of Brain Sciences, Imperial College, London, UK
| | - Martin M Reich
- Department of Neurology, University Hospital Wurzburg, Wuerzburg, Germany.
| |
Collapse
|
5
|
Hong R, Wu Z, Peng K, Zhang J, He Y, Zhang Z, Gao Y, Jin Y, Su X, Zhi H, Guan Q, Pan L, Jin L. Kinect-based objective assessment of the acute levodopa challenge test in parkinsonism: a feasibility study. Neurol Sci 2024; 45:2661-2670. [PMID: 38183553 DOI: 10.1007/s10072-023-07296-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/11/2023] [Accepted: 12/28/2023] [Indexed: 01/08/2024]
Abstract
INTRODUCTION The acute levodopa challenge test (ALCT) is an important and valuable examination but there are still some shortcomings with it. We aimed to objectively assess ALCT based on a depth camera and filter out the best indicators. METHODS Fifty-nine individuals with parkinsonism completed ALCT and the improvement rate (IR, which indicates the change in value before and after levodopa administration) of the Movement Disorder Society-Sponsored Revision of the Unified Parkinson's Disease Rating Scale part III (MDS-UPDRS III) was calculated. The kinematic features of the patients' movements in both the OFF and ON states were collected with an Azure Kinect depth camera. RESULTS The IR of MDS-UPDRS III was significantly correlated with the IRs of many kinematic features for arising from a chair, pronation-supination movements of the hand, finger tapping, toe tapping, leg agility, and gait (rs = - 0.277 ~ - 0.672, P < 0.05). Moderate to high discriminative values were found in the selected features in identifying a clinically significant response to levodopa with sensitivity, specificity, and area under the curve (AUC) in the range of 50-100%, 47.22%-97.22%, and 0.673-0.915, respectively. The resulting classifier combining kinematic features of toe tapping showed an excellent performance with an AUC of 0.966 (95% CI = 0.922-1.000, P < 0.001). The optimal cut-off value was 21.24% with sensitivity and specificity of 94.44% and 87.18%, respectively. CONCLUSION This study demonstrated the feasibility of measuring the effect of levodopa and objectively assessing ALCT based on kinematic data derived from an Azure Kinect-based system.
Collapse
Affiliation(s)
- Ronghua Hong
- Department of Neurology and Neurological Rehabilitation, Shanghai Disabled Persons' Federation Key Laboratory of Intelligent Rehabilitation Assistive Devices and Technologies, School of Medicine, Shanghai Yangzhi Rehabilitation Hospital (Shanghai Sunshine Rehabilitation Center), Tongji University, Shanghai, China
- Key Laboratory of Spine and Spinal Cord Injury Repair and Regeneration of Ministry of Education, Department of Neurology, School of Medicine, Neurotoxin Research CenterTongji HospitalTongji University, Shanghai, China
| | - Zhuang Wu
- Department of Neurology and Neurological Rehabilitation, Shanghai Disabled Persons' Federation Key Laboratory of Intelligent Rehabilitation Assistive Devices and Technologies, School of Medicine, Shanghai Yangzhi Rehabilitation Hospital (Shanghai Sunshine Rehabilitation Center), Tongji University, Shanghai, China
- Key Laboratory of Spine and Spinal Cord Injury Repair and Regeneration of Ministry of Education, Department of Neurology, School of Medicine, Neurotoxin Research CenterTongji HospitalTongji University, Shanghai, China
| | - Kangwen Peng
- Key Laboratory of Spine and Spinal Cord Injury Repair and Regeneration of Ministry of Education, Department of Neurology, School of Medicine, Neurotoxin Research CenterTongji HospitalTongji University, Shanghai, China
| | - Jingxing Zhang
- Key Laboratory of Spine and Spinal Cord Injury Repair and Regeneration of Ministry of Education, Department of Neurology, School of Medicine, Neurotoxin Research CenterTongji HospitalTongji University, Shanghai, China
| | - Yijing He
- Key Laboratory of Spine and Spinal Cord Injury Repair and Regeneration of Ministry of Education, Department of Neurology, School of Medicine, Neurotoxin Research CenterTongji HospitalTongji University, Shanghai, China
| | - Zhuoyu Zhang
- Key Laboratory of Spine and Spinal Cord Injury Repair and Regeneration of Ministry of Education, Department of Neurology, School of Medicine, Neurotoxin Research CenterTongji HospitalTongji University, Shanghai, China
| | - Yichen Gao
- IFLYTEK Suzhou Research Institute, Suzhou, China
| | - Yue Jin
- IFLYTEK Suzhou Research Institute, Suzhou, China
| | - Xiaoyun Su
- IFLYTEK Suzhou Research Institute, Suzhou, China
| | - Hongping Zhi
- IFLYTEK Suzhou Research Institute, Suzhou, China
| | - Qiang Guan
- Key Laboratory of Spine and Spinal Cord Injury Repair and Regeneration of Ministry of Education, Department of Neurology, School of Medicine, Neurotoxin Research CenterTongji HospitalTongji University, Shanghai, China
| | - Lizhen Pan
- Key Laboratory of Spine and Spinal Cord Injury Repair and Regeneration of Ministry of Education, Department of Neurology, School of Medicine, Neurotoxin Research CenterTongji HospitalTongji University, Shanghai, China.
| | - Lingjing Jin
- Department of Neurology and Neurological Rehabilitation, Shanghai Disabled Persons' Federation Key Laboratory of Intelligent Rehabilitation Assistive Devices and Technologies, School of Medicine, Shanghai Yangzhi Rehabilitation Hospital (Shanghai Sunshine Rehabilitation Center), Tongji University, Shanghai, China.
- Key Laboratory of Spine and Spinal Cord Injury Repair and Regeneration of Ministry of Education, Department of Neurology, School of Medicine, Neurotoxin Research CenterTongji HospitalTongji University, Shanghai, China.
- Collaborative Innovation Center for Brain Science, Tongji University, Shanghai, China.
| |
Collapse
|
6
|
Luiz LMD, Marques IA, Folador JP, Andrade AO. Intra and inter-rater remote assessment of bradykinesia in Parkinson's disease. Neurologia 2024; 39:345-352. [PMID: 38616062 DOI: 10.1016/j.nrleng.2021.08.006] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/07/2021] [Accepted: 08/04/2021] [Indexed: 04/16/2024] Open
Abstract
INTRODUCTION Reliable assessment of individuals with Parkinson's disease (PD) is essential for providing adequate treatment. Clinical assessment is a complex and time-consuming task, especially for bradykinesia, since its evaluation can be influenced by the degree of experience of the examiner, patient collaboration and individual bias. Improvement of the clinical evaluation can be obtained by considering assessments from several professionals. However, this is only true when inter and intra-rater agreement are high. Recently, the Movement Disorder Society highlighted, during the COVID-19 pandemic, the need to develop and validate technologies for remote assessment of the motor status of people with PD. Thus, this study introduces an objective strategy for the remote evaluation of bradykinesia using multi-specialist analysis. METHODS Twelve volunteers with PD participated and these were asked to execute finger tapping, hand opening/closing and pronation/supination movements. Each task was recorded and rated by fourteen PD health experts for each patient. The scores were assessed on an individual basis. Intra and inter-rater agreement and correlation were estimated. RESULTS The results showed that agreements and correlations between experienced examiners were high with low variability. In addition, group analysis was noted as possessing the potential to solve individual inconsistency bias. CONCLUSION Furthermore, this study demonstrated the need for a group with prior training and experience, along with indicating the importance for the development of a clinical protocol that can use telemedicine for the evaluation of individuals with PD, as well as the inclusion of a specialized mediating group. In Addition, this research helps to the development of a valid remote assessment of bradykinesia.
Collapse
Affiliation(s)
- L M D Luiz
- Centre for Innovation and Technology Assessment in Health, Faculty of Electrical Engineering, Federal University of Uberlândia, Uberlândia, Brazil.
| | - I A Marques
- Centre for Innovation and Technology Assessment in Health, Faculty of Electrical Engineering, Federal University of Uberlândia, Uberlândia, Brazil
| | - J P Folador
- Centre for Innovation and Technology Assessment in Health, Faculty of Electrical Engineering, Federal University of Uberlândia, Uberlândia, Brazil
| | - A O Andrade
- Centre for Innovation and Technology Assessment in Health, Faculty of Electrical Engineering, Federal University of Uberlândia, Uberlândia, Brazil
| |
Collapse
|
7
|
Alty J, Goldberg LR, Roccati E, Lawler K, Bai Q, Huang G, Bindoff AD, Li R, Wang X, St George RJ, Rudd K, Bartlett L, Collins JM, Aiyede M, Fernando N, Bhagwat A, Giffard J, Salmon K, McDonald S, King AE, Vickers JC. Development of a smartphone screening test for preclinical Alzheimer's disease and validation across the dementia continuum. BMC Neurol 2024; 24:127. [PMID: 38627686 PMCID: PMC11020184 DOI: 10.1186/s12883-024-03609-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/05/2024] [Accepted: 03/21/2024] [Indexed: 04/19/2024] Open
Abstract
BACKGROUND Dementia prevalence is predicted to triple to 152 million globally by 2050. Alzheimer's disease (AD) constitutes 70% of cases. There is an urgent need to identify individuals with preclinical AD, a 10-20-year period of progressive brain pathology without noticeable cognitive symptoms, for targeted risk reduction. Current tests of AD pathology are either too invasive, specialised or expensive for population-level assessments. Cognitive tests are normal in preclinical AD. Emerging evidence demonstrates that movement analysis is sensitive to AD across the disease continuum, including preclinical AD. Our new smartphone test, TapTalk, combines analysis of hand and speech-like movements to detect AD risk. This study aims to [1] determine which combinations of hand-speech movement data most accurately predict preclinical AD [2], determine usability, reliability, and validity of TapTalk in cognitively asymptomatic older adults and [3], prospectively validate TapTalk in older adults who have cognitive symptoms against cognitive tests and clinical diagnoses of Mild Cognitive Impairment and AD dementia. METHODS Aim 1 will be addressed in a cross-sectional study of at least 500 cognitively asymptomatic older adults who will complete computerised tests comprising measures of hand motor control (finger tapping) and oro-motor control (syllabic diadochokinesis). So far, 1382 adults, mean (SD) age 66.20 (7.65) years, range 50-92 (72.07% female) have been recruited. Motor measures will be compared to a blood-based AD biomarker, phosphorylated tau 181 to develop an algorithm that classifies preclinical AD risk. Aim 2 comprises three sub-studies in cognitively asymptomatic adults: (i) a cross-sectional study of 30-40 adults to determine the validity of data collection from different types of smartphones, (ii) a prospective cohort study of 50-100 adults ≥ 50 years old to determine usability and test-retest reliability, and (iii) a prospective cohort study of ~1,000 adults ≥ 50 years old to validate against cognitive measures. Aim 3 will be addressed in a cross-sectional study of ~200 participants with cognitive symptoms to validate TapTalk against Montreal Cognitive Assessment and interdisciplinary consensus diagnosis. DISCUSSION This study will establish the precision of TapTalk to identify preclinical AD and estimate risk of cognitive decline. If accurate, this innovative smartphone app will enable low-cost, accessible screening of individuals for AD risk. This will have wide applications in public health initiatives and clinical trials. TRIAL REGISTRATION ClinicalTrials.gov identifier: NCT06114914, 29 October 2023. Retrospectively registered.
Collapse
Affiliation(s)
- Jane Alty
- Wicking Dementia Research and Education Centre, University of Tasmania, Liverpool Street, Hobart, TAS, 7001, Australia.
- School of Medicine, University of Tasmania, Hobart, TAS, 7001, Australia.
- Royal Hobart Hospital, Hobart, TAS, 7001, Australia.
| | - Lynette R Goldberg
- Wicking Dementia Research and Education Centre, University of Tasmania, Liverpool Street, Hobart, TAS, 7001, Australia
| | - Eddy Roccati
- Wicking Dementia Research and Education Centre, University of Tasmania, Liverpool Street, Hobart, TAS, 7001, Australia
| | - Katherine Lawler
- Wicking Dementia Research and Education Centre, University of Tasmania, Liverpool Street, Hobart, TAS, 7001, Australia
- School of Allied Health, Human Services and Sport, La Trobe University, Melbourne, VIC, 3086, Australia
| | - Quan Bai
- School of Information and Communication Technology, University of Tasmania, Hobart, TAS, 7005, Australia
| | - Guan Huang
- Wicking Dementia Research and Education Centre, University of Tasmania, Liverpool Street, Hobart, TAS, 7001, Australia
| | - Aidan D Bindoff
- Wicking Dementia Research and Education Centre, University of Tasmania, Liverpool Street, Hobart, TAS, 7001, Australia
| | - Renjie Li
- Wicking Dementia Research and Education Centre, University of Tasmania, Liverpool Street, Hobart, TAS, 7001, Australia
- School of Information and Communication Technology, University of Tasmania, Hobart, TAS, 7005, Australia
| | - Xinyi Wang
- Wicking Dementia Research and Education Centre, University of Tasmania, Liverpool Street, Hobart, TAS, 7001, Australia
| | - Rebecca J St George
- School of Psychological Sciences, University of Tasmania, Hobart, TAS, 7005, Australia
| | - Kaylee Rudd
- Wicking Dementia Research and Education Centre, University of Tasmania, Liverpool Street, Hobart, TAS, 7001, Australia
| | - Larissa Bartlett
- Wicking Dementia Research and Education Centre, University of Tasmania, Liverpool Street, Hobart, TAS, 7001, Australia
| | - Jessica M Collins
- Wicking Dementia Research and Education Centre, University of Tasmania, Liverpool Street, Hobart, TAS, 7001, Australia
| | - Mimieveshiofuo Aiyede
- Wicking Dementia Research and Education Centre, University of Tasmania, Liverpool Street, Hobart, TAS, 7001, Australia
| | | | - Anju Bhagwat
- Royal Hobart Hospital, Hobart, TAS, 7001, Australia
| | - Julia Giffard
- Wicking Dementia Research and Education Centre, University of Tasmania, Liverpool Street, Hobart, TAS, 7001, Australia
| | - Katharine Salmon
- Wicking Dementia Research and Education Centre, University of Tasmania, Liverpool Street, Hobart, TAS, 7001, Australia
- Royal Hobart Hospital, Hobart, TAS, 7001, Australia
| | - Scott McDonald
- Wicking Dementia Research and Education Centre, University of Tasmania, Liverpool Street, Hobart, TAS, 7001, Australia
| | - Anna E King
- Wicking Dementia Research and Education Centre, University of Tasmania, Liverpool Street, Hobart, TAS, 7001, Australia
| | - James C Vickers
- Wicking Dementia Research and Education Centre, University of Tasmania, Liverpool Street, Hobart, TAS, 7001, Australia
| |
Collapse
|
8
|
Lam WWT, Fong KNK. Validity and Reliability of Upper Limb Kinematic Assessment Using a Markerless Motion Capture (MMC) System: A Pilot Study. Arch Phys Med Rehabil 2024; 105:673-681.e2. [PMID: 37981256 DOI: 10.1016/j.apmr.2023.10.018] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/20/2023] [Revised: 09/19/2023] [Accepted: 10/23/2023] [Indexed: 11/21/2023]
Abstract
OBJECTIVE To investigate the validity and test-retest reliability of a customized markerless motion capture (MMC) system that used iPad Pros with a Light Detection And Ranging scanner at two different viewing angles to measure the active range of motion (AROM) and the angular waveform of the upper-limb-joint angles of healthy adults performing functional tasks. DESIGN Participants were asked to perform shoulder and elbow actions for the investigator to take AROM measurements, followed by four tasks that simulated daily functioning. Each participant attended 2 experimental sessions, which were held at least 2 days and at most 14 days apart. SETTING A Vicon system and 2 iPad Pros installed with our MMC system were placed at 2 different angles to the participants and recorded their movements concurrently during each task. PARTICIPANTS Thirty healthy adults (mean age: 28.9, M/F ratio: 40/60). INTERVENTIONS Not applicable. MAIN OUTCOME MEASURES The AROM and the angular waveform of the upper-limb-joint angles. RESULTS The iPad Pro MMC system underestimated the shoulder joint and elbow joint angles in all four simulated functional tasks. The MMC demonstrated good to excellent test-retest reliability for the shoulder joint AROM measurements in all 4 tasks. CONCLUSIONS The maximal AROM measurements calculated by the MMC system had consistently smaller values than those measured by the goniometer. An MMC in iPad Pro system might not be able to replace conventional goniometry for clinical ROM measurements, but it is still suggested for use in home-based and telerehabilitation training for intra-subject measurements because of its good reliability, low cost, and portability. Further development to improve its performance in motion capture and analysis in disease populations is warranted.
Collapse
Affiliation(s)
- Winnie W T Lam
- Department of Rehabilitation Sciences, The Hong Kong Polytechnic University, Kowloon, Hong Kong SAR.
| | - Kenneth N K Fong
- Department of Rehabilitation Sciences, The Hong Kong Polytechnic University, Kowloon, Hong Kong SAR
| |
Collapse
|
9
|
Wong DC, Williams S. Artificial intelligence analysis of videos to augment clinical assessment: an overview. Neural Regen Res 2024; 19:717-718. [PMID: 37843200 PMCID: PMC10664118 DOI: 10.4103/1673-5374.382249] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/30/2023] [Revised: 05/30/2023] [Accepted: 06/27/2023] [Indexed: 10/17/2023] Open
Affiliation(s)
- David C. Wong
- University of Leeds; Stefan Williams, Leeds Teaching Hospitals Trust, Leeds, UK
| | - Stefan Williams
- University of Leeds; Stefan Williams, Leeds Teaching Hospitals Trust, Leeds, UK
| |
Collapse
|
10
|
Park KW, Mirian MS, McKeown MJ. Artificial intelligence-based video monitoring of movement disorders in the elderly: a review on current and future landscapes. Singapore Med J 2024; 65:141-149. [PMID: 38527298 PMCID: PMC11060643 DOI: 10.4103/singaporemedj.smj-2023-189] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/29/2023] [Accepted: 12/19/2023] [Indexed: 03/27/2024]
Abstract
ABSTRACT Due to global ageing, the burden of chronic movement and neurological disorders (Parkinson's disease and essential tremor) is rapidly increasing. Current diagnosis and monitoring of these disorders rely largely on face-to-face assessments utilising clinical rating scales, which are semi-subjective and time-consuming. To address these challenges, the utilisation of artificial intelligence (AI) has emerged. This review explores the advantages and challenges associated with using AI-driven video monitoring to care for elderly patients with movement disorders. The AI-based video monitoring systems offer improved efficiency and objectivity in remote patient monitoring, enabling real-time analysis of data, more uniform outcomes and augmented support for clinical trials. However, challenges, such as video quality, privacy compliance and noisy training labels, during development need to be addressed. Ultimately, the advancement of video monitoring for movement disorders is expected to evolve towards discreet, home-based evaluations during routine daily activities. This progression must incorporate data security, ethical considerations and adherence to regulatory standards.
Collapse
Affiliation(s)
- Kye Won Park
- Pacific Parkinson Research Centre, University of British Columbia, Vancouver, British Columbia, Canada
| | - Maryam S Mirian
- Pacific Parkinson Research Centre, University of British Columbia, Vancouver, British Columbia, Canada
| | - Martin J McKeown
- Pacific Parkinson Research Centre, University of British Columbia, Vancouver, British Columbia, Canada
- Department of Medicine (Neurology), University of British Columbia, Vancouver, British Columbia, Canada
| |
Collapse
|
11
|
Cuk A, Bezdan T, Jovanovic L, Antonijevic M, Stankovic M, Simic V, Zivkovic M, Bacanin N. Tuning attention based long-short term memory neural networks for Parkinson's disease detection using modified metaheuristics. Sci Rep 2024; 14:4309. [PMID: 38383690 PMCID: PMC10881563 DOI: 10.1038/s41598-024-54680-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/24/2023] [Accepted: 02/15/2024] [Indexed: 02/23/2024] Open
Abstract
Parkinson's disease (PD) is a progressively debilitating neurodegenerative disorder that primarily affects the dopaminergic system in the basal ganglia, impacting millions of individuals globally. The clinical manifestations of the disease include resting tremors, muscle rigidity, bradykinesia, and postural instability. Diagnosis relies mainly on clinical evaluation, lacking reliable diagnostic tests and being inherently imprecise and subjective. Early detection of PD is crucial for initiating treatments that, while unable to cure the chronic condition, can enhance the life quality of patients and alleviate symptoms. This study explores the potential of utilizing long-short term memory neural networks (LSTM) with attention mechanisms to detect Parkinson's disease based on dual-task walking test data. Given that the performance of networks is significantly inductance by architecture and training parameter choices, a modified version of the recently introduced crayfish optimization algorithm (COA) is proposed, specifically tailored to the requirements of this investigation. The proposed optimizer is assessed on a publicly accessible real-world clinical gait in Parkinson's disease dataset, and the results demonstrate its promise, achieving an accuracy of 87.4187 % for the best-constructed models.
Collapse
Affiliation(s)
- Aleksa Cuk
- Singidunum University, Danijelova 32, Belgrade, 11010, Serbia
| | - Timea Bezdan
- Singidunum University, Danijelova 32, Belgrade, 11010, Serbia
| | - Luka Jovanovic
- Singidunum University, Danijelova 32, Belgrade, 11010, Serbia
| | | | - Milos Stankovic
- Singidunum University, Danijelova 32, Belgrade, 11010, Serbia
| | - Vladimir Simic
- Faculty of Transport and Traffic Engineering, University of Belgrade, Vojvode Stepe 305, Belgrade, 11010, Serbia
- College of Engineering, Department of Industrial Engineering and Management, Yuan Ze University, Taoyuan City, 320315, Taiwan
- College of Informatics, Korea University, 145, Anam-ro, Seongbuk-gu, Seoul, Republic of Korea
| | | | - Nebojsa Bacanin
- Singidunum University, Danijelova 32, Belgrade, 11010, Serbia.
- MEU Research Unit, Middle East University, Amman, Jordan.
- Faculty of Data Science and Information Technology, INTI International University, 71800, Nilai, Malaysia.
| |
Collapse
|
12
|
Amo-Salas J, Olivares-Gil A, García-Bustillo Á, García-García D, Arnaiz-González Á, Cubo E. Computer Vision for Parkinson's Disease Evaluation: A Survey on Finger Tapping. Healthcare (Basel) 2024; 12:439. [PMID: 38391815 PMCID: PMC10888014 DOI: 10.3390/healthcare12040439] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/22/2023] [Revised: 02/01/2024] [Accepted: 02/06/2024] [Indexed: 02/24/2024] Open
Abstract
Parkinson's disease (PD) is a progressive neurodegenerative disorder whose prevalence has steadily been rising over the years. Specialist neurologists across the world assess and diagnose patients with PD, although the diagnostic process is time-consuming and various symptoms take years to appear, which means that the diagnosis is prone to human error. The partial automatization of PD assessment and diagnosis through computational processes has therefore been considered for some time. One well-known tool for PD assessment is finger tapping (FT), which can now be assessed through computer vision (CV). Artificial intelligence and related advances over recent decades, more specifically in the area of CV, have made it possible to develop computer systems that can help specialists assess and diagnose PD. The aim of this study is to review some advances related to CV techniques and FT so as to offer insight into future research lines that technological advances are now opening up.
Collapse
Affiliation(s)
- Javier Amo-Salas
- Escuela Politécnica Superior, Departamento de Ingeniería Informática, Universidad de Burgos, 09001 Burgos, Spain
| | - Alicia Olivares-Gil
- Escuela Politécnica Superior, Departamento de Ingeniería Informática, Universidad de Burgos, 09001 Burgos, Spain
| | - Álvaro García-Bustillo
- Facultad de Ciencias de la Salud, Departamento de Ciencias de la Salud, Universidad de Burgos, 09001 Burgos, Spain
| | - David García-García
- Escuela Politécnica Superior, Departamento de Ingeniería Informática, Universidad de Burgos, 09001 Burgos, Spain
| | - Álvar Arnaiz-González
- Escuela Politécnica Superior, Departamento de Ingeniería Informática, Universidad de Burgos, 09001 Burgos, Spain
| | - Esther Cubo
- Servicio de Neurología, Hospital Universitario de Burgos, 09006 Burgos, Spain
| |
Collapse
|
13
|
Yang YY, Ho MY, Tai CH, Wu RM, Kuo MC, Tseng YJ. FastEval Parkinsonism: an instant deep learning-assisted video-based online system for Parkinsonian motor symptom evaluation. NPJ Digit Med 2024; 7:31. [PMID: 38332372 PMCID: PMC10853559 DOI: 10.1038/s41746-024-01022-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/27/2023] [Accepted: 01/18/2024] [Indexed: 02/10/2024] Open
Abstract
The Motor Disorder Society's Unified Parkinson's Disease Rating Scale (MDS-UPDRS) is designed to assess bradykinesia, the cardinal symptoms of Parkinson's disease (PD). However, it cannot capture the all-day variability of bradykinesia outside the clinical environment. Here, we introduce FastEval Parkinsonism ( https://fastevalp.cmdm.tw/ ), a deep learning-driven video-based system, providing users to capture keypoints, estimate the severity, and summarize in a report. Leveraging 840 finger-tapping videos from 186 individuals (103 patients with Parkinson's disease (PD), 24 participants with atypical parkinsonism (APD), 12 elderly with mild parkinsonism signs (MPS), and 47 healthy controls (HCs)), we employ a dilated convolution neural network with two data augmentation techniques. Our model achieves acceptable accuracies (AAC) of 88.0% and 81.5%. The frequency-intensity (FI) value of thumb-index finger distance was indicated as a pivotal hand parameter to quantify the performance. Our model also shows the usability for multi-angle videos, tested in an external database enrolling over 300 PD patients.
Collapse
Affiliation(s)
- Yu-Yuan Yang
- Graduate Institute of Biomedical Electronics and Bioinformatics, National Taiwan University, No. 1 Roosevelt Rd. Sec. 4, Taipei, 10617, Taiwan, ROC
| | - Ming-Yang Ho
- Department of Computer Science and Information Engineering, National Taiwan University, No. 1 Roosevelt Rd. Sec. 4, Taipei, 10617, Taiwan, ROC
| | - Chung-Hwei Tai
- Department of Neurology, National Taiwan University Hospital, No. 1 Changde St., Zhongzheng Dist., Taipei City, 100229, Taiwan, ROC
| | - Ruey-Meei Wu
- Department of Medicine, National Taiwan University Cancer Center, No. 57, Lane 155, Sec. 3, Keelung Rd., Da'an Dist., Taipei City, 106, Taiwan, ROC
| | - Ming-Che Kuo
- Department of Neurology, National Taiwan University Hospital, No. 1 Changde St., Zhongzheng Dist., Taipei City, 100229, Taiwan, ROC.
- Department of Medicine, National Taiwan University Cancer Center, No. 57, Lane 155, Sec. 3, Keelung Rd., Da'an Dist., Taipei City, 106, Taiwan, ROC.
| | - Yufeng Jane Tseng
- Graduate Institute of Biomedical Electronics and Bioinformatics, National Taiwan University, No. 1 Roosevelt Rd. Sec. 4, Taipei, 10617, Taiwan, ROC.
- Department of Computer Science and Information Engineering, National Taiwan University, No. 1 Roosevelt Rd. Sec. 4, Taipei, 10617, Taiwan, ROC.
| |
Collapse
|
14
|
Li R, Wang X, Lawler K, Garg S, St George RJ, Bindoff AD, Bartlett L, Roccati E, King AE, Vickers JC, Bai Q, Alty J. Brief webcam test of hand movements predicts episodic memory, executive function, and working memory in a community sample of cognitively asymptomatic older adults. ALZHEIMER'S & DEMENTIA (AMSTERDAM, NETHERLANDS) 2024; 16:e12520. [PMID: 38274411 PMCID: PMC10809289 DOI: 10.1002/dad2.12520] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 03/23/2023] [Revised: 12/01/2023] [Accepted: 12/05/2023] [Indexed: 01/27/2024]
Abstract
INTRODUCTION Low-cost simple tests for preclinical Alzheimer's disease are a research priority. We evaluated whether remote unsupervised webcam recordings of finger-tapping were associated with cognitive performance in older adults. METHODS A total of 404 cognitively-asymptomatic participants (64.6 [6.77] years; 70.8% female) completed 10-second finger-tapping tests (Tasmanian [TAS] Test) and cognitive tests (Cambridge Neuropsychological Test Automated Battery [CANTAB]) online at home. Regression models including hand movement features were compared with null models (comprising age, sex, and education level); change in Akaike Information Criterion greater than 2 (ΔAIC > 2) denoted statistical difference. RESULTS Hand movement features improved prediction of episodic memory, executive function, and working memory scores (ΔAIC > 2). Dominant hand features outperformed nondominant hand features for episodic memory (ΔAIC = 2.5), executive function (ΔAIC = 4.8), and working memory (ΔAIC = 2.2). DISCUSSION This brief webcam test improved prediction of cognitive performance compared to age, sex, and education. Finger-tapping holds potential as a remote language-agnostic screening tool to stratify community cohorts at risk for cognitive decline.
Collapse
Affiliation(s)
- Renjie Li
- Wicking Dementia Research and Education CentreUniversity of TasmaniaHobartTasmaniaAustralia
- School of ICTUniversity of TasmaniaHobartTasmaniaAustralia
| | - Xinyi Wang
- Wicking Dementia Research and Education CentreUniversity of TasmaniaHobartTasmaniaAustralia
| | - Katherine Lawler
- Wicking Dementia Research and Education CentreUniversity of TasmaniaHobartTasmaniaAustralia
- School of Allied HealthHuman Services and SportLa Trobe UniversityMelbourneVictoriaAustralia
| | - Saurabh Garg
- School of ICTUniversity of TasmaniaHobartTasmaniaAustralia
| | | | - Aidan D. Bindoff
- Wicking Dementia Research and Education CentreUniversity of TasmaniaHobartTasmaniaAustralia
| | - Larissa Bartlett
- Wicking Dementia Research and Education CentreUniversity of TasmaniaHobartTasmaniaAustralia
| | - Eddy Roccati
- Wicking Dementia Research and Education CentreUniversity of TasmaniaHobartTasmaniaAustralia
| | - Anna E. King
- Wicking Dementia Research and Education CentreUniversity of TasmaniaHobartTasmaniaAustralia
| | - James C. Vickers
- Wicking Dementia Research and Education CentreUniversity of TasmaniaHobartTasmaniaAustralia
| | - Quan Bai
- School of ICTUniversity of TasmaniaHobartTasmaniaAustralia
| | - Jane Alty
- Wicking Dementia Research and Education CentreUniversity of TasmaniaHobartTasmaniaAustralia
- School of MedicineUniversity of TasmaniaHobartTasmaniaAustralia
- Neurology DepartmentRoyal Hobart HospitalHobartTasmaniaAustralia
| |
Collapse
|
15
|
Park R, Lee DH, Koh CS, Kwon YW, Chae SY, Kim C, Jung HH, Jeong J, Hong SW. Laser-Assisted Structuring of Graphene Films with Biocompatible Liquid Crystal Polymer for Skin/Brain-Interfaced Electrodes. Adv Healthc Mater 2024; 13:e2301753. [PMID: 37820714 PMCID: PMC11468237 DOI: 10.1002/adhm.202301753] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/02/2023] [Revised: 10/09/2023] [Indexed: 10/13/2023]
Abstract
The work presented here introduces a facile strategy for the development of flexible and stretchable electrodes that harness the robust characteristics of carbon nanomaterials through laser processing techniques on a liquid crystal polymer (LCP) film. By utilizing LCP film as a biocompatible electronic substrate, control is demonstrated over the laser irradiation parameters to achieve efficient pattern generation and transfer printing processes, thereby yielding highly conductive laser-induced graphene (LIG) bioelectrodes. To enhance the resolution of the patterned LIG film, shadow masks are employed during laser scanning on the LCP film surface. This approach is compatible with surface-mounted device integration, enabling the circuit writing of LIG/LCP materials in a flexible format. Moreover, kirigami-inspired on-skin bioelectrodes are introduced that exhibit reasonable stretchability, enabling independent connections to healthcare hardware platforms for electrocardiogram (ECG) and electromyography (EMG) measurements. Additionally, a brain-interfaced LIG microelectrode array is proposed that combines mechanically compliant architectures with LCP encapsulation for stimulation and recording purposes, leveraging their advantageous structural features and superior electrochemical properties. This developed approach offers a cost-effective and scalable route for producing patterned arrays of laser-converted graphene as bioelectrodes. These bioelectrodes serve as ideal circuit-enabled flexible substrates with long-term reliability in the ionic environment of the human body.
Collapse
Affiliation(s)
- Rowoon Park
- Department of Optics and Mechatronics Engineering, Department of Cogno‐Mechatronics Engineering, College of Nanoscience and NanotechnologyPusan National UniversityBusan46241Republic of Korea
| | - Dong Hyeon Lee
- School of Mechanical EngineeringPusan National UniversityBusan46241Republic of Korea
| | - Chin Su Koh
- Department of NeurosurgeryCollege of MedicineYonsei UniversitySeoul03722Republic of Korea
| | - Young Woo Kwon
- Engineering Research Center for Color‐Modulated Extra‐Sensory Perception TechnologyPusan National UniversityBusan46241Republic of Korea
| | - Seon Yeong Chae
- Engineering Research Center for Color‐Modulated Extra‐Sensory Perception TechnologyPusan National UniversityBusan46241Republic of Korea
| | - Chang‐Seok Kim
- Department of Optics and Mechatronics Engineering, Department of Cogno‐Mechatronics Engineering, College of Nanoscience and NanotechnologyPusan National UniversityBusan46241Republic of Korea
- Engineering Research Center for Color‐Modulated Extra‐Sensory Perception TechnologyPusan National UniversityBusan46241Republic of Korea
| | - Hyun Ho Jung
- Department of NeurosurgeryCollege of MedicineYonsei UniversitySeoul03722Republic of Korea
| | - Joonsoo Jeong
- School of Biomedical Convergence EngineeringDepartment of Information Convergence EngineeringPusan National UniversityYangsan50612Republic of Korea
| | - Suck Won Hong
- Department of Optics and Mechatronics Engineering, Department of Cogno‐Mechatronics Engineering, College of Nanoscience and NanotechnologyPusan National UniversityBusan46241Republic of Korea
- Engineering Research Center for Color‐Modulated Extra‐Sensory Perception TechnologyPusan National UniversityBusan46241Republic of Korea
| |
Collapse
|
16
|
Conelea C, Liang H, DuBois M, Raab B, Kellman M, Wellen B, Jacob S, Wang S, Sun J, Lim K. Automated Quantification of Eye Tics Using Computer Vision and Deep Learning Techniques. Mov Disord 2024; 39:183-191. [PMID: 38146055 PMCID: PMC10895867 DOI: 10.1002/mds.29593] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/03/2023] [Revised: 08/04/2023] [Accepted: 08/10/2023] [Indexed: 12/27/2023] Open
Abstract
BACKGROUND Tourette syndrome (TS) tics are typically quantified using "paper and pencil" rating scales that are susceptible to factors that adversely impact validity. Video-based methods to more objectively quantify tics have been developed but are challenged by reliance on human raters and procedures that are resource intensive. Computer vision approaches that automate detection of atypical movements may be useful to apply to tic quantification. OBJECTIVE The current proof-of-concept study applied a computer vision approach to train a supervised deep learning algorithm to detect eye tics in video, the most common tic type in patients with TS. METHODS Videos (N = 54) of 11 adolescent patients with TS were rigorously coded by trained human raters to identify 1.5-second clips depicting "eye tic events" (N = 1775) and "non-tic events" (N = 3680). Clips were encoded into three-dimensional facial landmarks. Supervised deep learning was applied to processed data using random split and disjoint split regimens to simulate model validity under different conditions. RESULTS Area under receiver operating characteristic curve was 0.89 for the random split regimen, indicating high accuracy in the algorithm's ability to properly classify eye tic vs. non-eye tic movements. Area under receiver operating characteristic curve was 0.74 for the disjoint split regimen, suggesting that algorithm generalizability is more limited when trained on a small patient sample. CONCLUSIONS The algorithm was successful in detecting eye tics in unseen validation sets. Automated tic detection from video is a promising approach for tic quantification that may have future utility in TS screening, diagnostics, and treatment outcome measurement. © 2023 The Authors. Movement Disorders published by Wiley Periodicals LLC on behalf of International Parkinson and Movement Disorder Society.
Collapse
Affiliation(s)
- Christine Conelea
- University of Minnesota, Department of Psychiatry & Behavioral Sciences
| | - Hengyue Liang
- University of Minnesota, Department of Electrical & Computer Engineering
| | - Megan DuBois
- University of Minnesota, Department of Psychiatry & Behavioral Sciences
| | - Brittany Raab
- University of Minnesota, Department of Psychiatry & Behavioral Sciences
| | - Mia Kellman
- University of Minnesota, Department of Psychiatry & Behavioral Sciences
| | - Brianna Wellen
- University of Minnesota, Department of Psychiatry & Behavioral Sciences
| | - Suma Jacob
- University of Minnesota, Department of Psychiatry & Behavioral Sciences
| | - Sonya Wang
- University of Minnesota, Department of Neurology
| | - Ju Sun
- University of Minnesota, Department of Computer Science & Engineering
| | - Kelvin Lim
- University of Minnesota, Department of Psychiatry & Behavioral Sciences
| |
Collapse
|
17
|
Park S, Yoo HJ, Jang JS, Lee SH. Automated non-contact measurement of the spine curvature at the sagittal plane using a deep neural network. Clin Biomech (Bristol, Avon) 2024; 111:106146. [PMID: 37976690 DOI: 10.1016/j.clinbiomech.2023.106146] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/24/2023] [Revised: 10/28/2023] [Accepted: 11/08/2023] [Indexed: 11/19/2023]
Abstract
BACKGROUND Non-radiographical techniques have been suggested to measure the spine curvature at the sagittal plane. However, a neural network has not been used to measure the curvature. METHODS A single video camera captured images of a standing posture at the sagittal plane from twenty healthy males. Six marker positions along the spine's contour in each image were identified for measuring inclination, thoracic kyphosis, and lumbar lordosis angles. We estimated three inflection points around the neck, hip, and between the neck and hip, followed by identifying two adjacent marker positions per inflection point to compute its tangent. The angular deviation of each tangent line from the horizontal was computed to measure inclination angles. Thoracic kyphosis and lumbar lordosis angles were computed by the angular difference between the two adjacent tangents. A deep neural network was trained with 500,000 iterations using the labeled images from 18 participants (388 and 44 images for training and test set) and then evaluated using the unseen images (2 participants, 48 images; evaluation set). FINDINGS The mean total training and test errors were <2 pixels (∼ 0.6 cm). The total error in the evaluation set was qualitatively comparable (∼ 3 pixels = ∼ 0.9 cm), suggesting the model performance was maintained in the unseen data. The angle values between labeled and network-predicted marker positions were similar in the evaluation set. INTERPRETATION The network training with a relatively small number of images was successful based on the small error values observed in the evaluation set. The model may be an affordable, automated, and non-contact measurement tool for the human spine curvature.
Collapse
Affiliation(s)
- Sangsoo Park
- School of Global Sport Studies, Korea University Sejong Campus, Sejong City 30019, South Korea.
| | - Hyun-Joon Yoo
- Korea University Research Institute for Medical Bigdata Science, Korea University, Goryeodae-ro 73, Seongbuk-gu, Seoul 02841, South Korea
| | - Jin Su Jang
- Human Behavior & Genetic Institute, Associate Research Center, Korea University, Goryeodae-ro 73, Seongbuk-gu, Seoul 02841, South Korea
| | - Sang-Heon Lee
- Department of Physical Medicine and Rehabilitation, Korea University Anam Hospital, Korea University College of Medicine, Goryeodae-ro 73, Seongbuk-gu, Seoul 02841, South Korea
| |
Collapse
|
18
|
Rudd KD, Lawler K, Callisaya ML, Alty J. Investigating the associations between upper limb motor function and cognitive impairment: a scoping review. GeroScience 2023; 45:3449-3473. [PMID: 37337026 PMCID: PMC10643613 DOI: 10.1007/s11357-023-00844-z] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/30/2023] [Accepted: 05/26/2023] [Indexed: 06/21/2023] Open
Abstract
Upper limb motor function is a potential new biomarker of cognitive impairment and may aid discrimination from healthy ageing. However, it remains unclear which assessments to use. This study aimed to explore what methods have been used and to describe associations between upper limb function and cognitive impairment. A scoping review was conducted using PubMed, CINAHL and Web of Science. A systematic search was undertaken, including synonyms for key concepts 'upper limb', 'motor function' and 'cognitive impairment'. Selection criteria included tests of upper limb motor function and impaired cognition in adults. Analysis was by narrative synthesis. Sixty papers published between 1998 and 2022, comprising 41,800 participants, were included. The most common assessment tasks were finger tapping, Purdue Pegboard Test and functional tasks such as writing. Protocols were diverse in terms of equipment used and recording duration. Most participants were recruited from clinical settings. Alzheimer's Disease was the most common cause of cognitive impairment. Results were mixed but, generally, slower speed, more errors, and greater variability in upper limb movement variables was associated with cognitive impairment. This review maps the upper limb motor function assessments used and summarises the available evidence on how these associate with cognitive impairment. It identifies research gaps and may help guide protocols for future research. There is potential for upper limb motor function to be used in assessments of cognitive impairment.
Collapse
Affiliation(s)
- Kaylee D Rudd
- Wicking Dementia Research and Education Centre, University of Tasmania, Hobart, Tasmania, Australia
| | - Katherine Lawler
- Wicking Dementia Research and Education Centre, University of Tasmania, Hobart, Tasmania, Australia
- School of Allied Health, Human Services and Sport, La Trobe University, Melbourne, Victoria, Australia
| | - Michele L Callisaya
- Menzies Institute for Medical Research, University of Tasmania, Hobart, Tasmania, Australia
- Peninsula Clinical School, Monash University, Melbourne, Victoria, Australia
| | - Jane Alty
- Wicking Dementia Research and Education Centre, University of Tasmania, Hobart, Tasmania, Australia.
- School of Medicine, University of Tasmania, Hobart, Tasmania, Australia.
- Neurology Department, Royal Hobart Hospital, Hobart, Tasmania, Australia.
| |
Collapse
|
19
|
Prakash P, Kaur R, Levy J, Sowers R, Brasic J, Hernandez ME. A Deep Learning Approach for Grading of Motor Impairment Severity in Parkinson's Disease. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2023; 2023:1-4. [PMID: 38083387 DOI: 10.1109/embc40787.2023.10341122] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/18/2023]
Abstract
Objective and quantitative monitoring of movement impairments is crucial for detecting progression in neurological conditions such as Parkinson's disease (PD). This study examined the ability of deep learning approaches to grade motor impairment severity in a modified version of the Movement Disorders Society-sponsored revision of the Unified Parkinson's Disease Rating Scale (MDS-UPDRS) using low-cost wearable sensors. A convolutional neural network architecture, XceptionTime, was used to classify lower and higher levels of motor impairment in persons with PD, across five distinct rhythmic tasks: finger tapping, hand movements, pronation-supination movements of the hands, toe tapping, and leg agility. In addition, an aggregate model was trained on data from all tasks together for evaluating bradykinesia symptom severity in PD. The model performance was highest in the hand movement tasks with an accuracy of 82.6% in the hold-out test dataset; the accuracy for the aggregate model was 79.7%, however, it demonstrated the lowest variability. Overall, these findings suggest the feasibility of integrating low-cost wearable technology and deep learning approaches to automatically and objectively quantify motor impairment in persons with PD. This approach may provide a viable solution for a widely deployable telemedicine solution.
Collapse
|
20
|
Habets JGV, Spooner RK, Mathiopoulou V, Feldmann LK, Busch JL, Roediger J, Bahners BH, Schnitzler A, Florin E, Kühn AA. A First Methodological Development and Validation of ReTap: An Open-Source UPDRS Finger Tapping Assessment Tool Based on Accelerometer-Data. SENSORS (BASEL, SWITZERLAND) 2023; 23:s23115238. [PMID: 37299968 DOI: 10.3390/s23115238] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/28/2023] [Revised: 05/24/2023] [Accepted: 05/29/2023] [Indexed: 06/12/2023]
Abstract
Bradykinesia is a cardinal hallmark of Parkinson's disease (PD). Improvement in bradykinesia is an important signature of effective treatment. Finger tapping is commonly used to index bradykinesia, albeit these approaches largely rely on subjective clinical evaluations. Moreover, recently developed automated bradykinesia scoring tools are proprietary and are not suitable for capturing intraday symptom fluctuation. We assessed finger tapping (i.e., Unified Parkinson's Disease Rating Scale (UPDRS) item 3.4) in 37 people with Parkinson's disease (PwP) during routine treatment follow ups and analyzed their 350 sessions of 10-s tapping using index finger accelerometry. Herein, we developed and validated ReTap, an open-source tool for the automated prediction of finger tapping scores. ReTap successfully detected tapping blocks in over 94% of cases and extracted clinically relevant kinematic features per tap. Importantly, based on the kinematic features, ReTap predicted expert-rated UPDRS scores significantly better than chance in a hold out validation sample (n = 102). Moreover, ReTap-predicted UPDRS scores correlated positively with expert ratings in over 70% of the individual subjects in the holdout dataset. ReTap has the potential to provide accessible and reliable finger tapping scores, either in the clinic or at home, and may contribute to open-source and detailed analyses of bradykinesia.
Collapse
Affiliation(s)
- Jeroen G V Habets
- Movement Disorder and Neuromodulation Unit, Department of Neurology, Charité Universitaetsmedizin Berlin, 10117 Berlin, Germany
| | - Rachel K Spooner
- Institute of Clinical Neuroscience and Medical Psychology, Medical Faculty, Heinrich Heine University Düsseldorf, 40225 Düsseldorf, Germany
| | - Varvara Mathiopoulou
- Movement Disorder and Neuromodulation Unit, Department of Neurology, Charité Universitaetsmedizin Berlin, 10117 Berlin, Germany
| | - Lucia K Feldmann
- Movement Disorder and Neuromodulation Unit, Department of Neurology, Charité Universitaetsmedizin Berlin, 10117 Berlin, Germany
| | - Johannes L Busch
- Movement Disorder and Neuromodulation Unit, Department of Neurology, Charité Universitaetsmedizin Berlin, 10117 Berlin, Germany
| | - Jan Roediger
- Movement Disorder and Neuromodulation Unit, Department of Neurology, Charité Universitaetsmedizin Berlin, 10117 Berlin, Germany
| | - Bahne H Bahners
- Institute of Clinical Neuroscience and Medical Psychology, Medical Faculty, Heinrich Heine University Düsseldorf, 40225 Düsseldorf, Germany
- Department of Neurology, Center for Movement Disorders and Neuromodulation, Medical Faculty, Heinrich-Heine University Düsseldorf, 40225 Düsseldorf, Germany
| | - Alfons Schnitzler
- Institute of Clinical Neuroscience and Medical Psychology, Medical Faculty, Heinrich Heine University Düsseldorf, 40225 Düsseldorf, Germany
- Department of Neurology, Center for Movement Disorders and Neuromodulation, Medical Faculty, Heinrich-Heine University Düsseldorf, 40225 Düsseldorf, Germany
| | - Esther Florin
- Institute of Clinical Neuroscience and Medical Psychology, Medical Faculty, Heinrich Heine University Düsseldorf, 40225 Düsseldorf, Germany
| | - Andrea A Kühn
- Movement Disorder and Neuromodulation Unit, Department of Neurology, Charité Universitaetsmedizin Berlin, 10117 Berlin, Germany
| |
Collapse
|
21
|
Lam WWT, Tang YM, Fong KNK. A systematic review of the applications of markerless motion capture (MMC) technology for clinical measurement in rehabilitation. J Neuroeng Rehabil 2023; 20:57. [PMID: 37131238 PMCID: PMC10155325 DOI: 10.1186/s12984-023-01186-9] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/04/2022] [Accepted: 04/26/2023] [Indexed: 05/04/2023] Open
Abstract
BACKGROUND Markerless motion capture (MMC) technology has been developed to avoid the need for body marker placement during motion tracking and analysis of human movement. Although researchers have long proposed the use of MMC technology in clinical measurement-identification and measurement of movement kinematics in a clinical population, its actual application is still in its preliminary stages. The benefits of MMC technology are also inconclusive with regard to its use in assessing patients' conditions. In this review we put a minor focus on the method's engineering components and sought primarily to determine the current application of MMC as a clinical measurement tool in rehabilitation. METHODS A systematic computerized literature search was conducted in PubMed, Medline, CINAHL, CENTRAL, EMBASE, and IEEE. The search keywords used in each database were "Markerless Motion Capture OR Motion Capture OR Motion Capture Technology OR Markerless Motion Capture Technology OR Computer Vision OR Video-based OR Pose Estimation AND Assessment OR Clinical Assessment OR Clinical Measurement OR Assess." Only peer-reviewed articles that applied MMC technology for clinical measurement were included. The last search took place on March 6, 2023. Details regarding the application of MMC technology for different types of patients and body parts, as well as the assessment results, were summarized. RESULTS A total of 65 studies were included. The MMC systems used for measurement were most frequently used to identify symptoms or to detect differences in movement patterns between disease populations and their healthy counterparts. Patients with Parkinson's disease (PD) who demonstrated obvious and well-defined physical signs were the largest patient group to which MMC assessment had been applied. Microsoft Kinect was the most frequently used MMC system, although there was a recent trend of motion analysis using video captured with a smartphone camera. CONCLUSIONS This review explored the current uses of MMC technology for clinical measurement. MMC technology has the potential to be used as an assessment tool as well as to assist in the detection and identification of symptoms, which might further contribute to the use of an artificial intelligence method for early screening for diseases. Further studies are warranted to develop and integrate MMC system in a platform that can be user-friendly and accurately analyzed by clinicians to extend the use of MMC technology in the disease populations.
Collapse
Affiliation(s)
- Winnie W. T. Lam
- Department of Rehabilitation Sciences, The Hong Kong Polytechnic University, Kowloon, Hong Kong SAR China
| | - Yuk Ming Tang
- Department of Industrial and Systems Engineering, The Hong Kong Polytechnic University, Kowloon, Hong Kong SAR China
| | - Kenneth N. K. Fong
- Department of Rehabilitation Sciences, The Hong Kong Polytechnic University, Kowloon, Hong Kong SAR China
| |
Collapse
|
22
|
Morinan G, Dushin Y, Sarapata G, Rupprechter S, Peng Y, Girges C, Salazar M, Milabo C, Sibley K, Foltynie T, Cociasu I, Ricciardi L, Baig F, Morgante F, Leyland LA, Weil RS, Gilron R, O’Keeffe J. Computer vision quantification of whole-body Parkinsonian bradykinesia using a large multi-site population. NPJ Parkinsons Dis 2023; 9:10. [PMID: 36707523 PMCID: PMC9883391 DOI: 10.1038/s41531-023-00454-8] [Citation(s) in RCA: 14] [Impact Index Per Article: 14.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/09/2022] [Accepted: 01/13/2023] [Indexed: 01/28/2023] Open
Abstract
Parkinson's disease (PD) is a common neurological disorder, with bradykinesia being one of its cardinal features. Objective quantification of bradykinesia using computer vision has the potential to standardise decision-making, for patient treatment and clinical trials, while facilitating remote assessment. We utilised a dataset of part-3 MDS-UPDRS motor assessments, collected at four independent clinical and one research sites on two continents, to build computer-vision-based models capable of inferring the correct severity rating robustly and consistently across all identifiable subgroups of patients. These results contrast with previous work limited by small sample sizes and small numbers of sites. Our bradykinesia estimation corresponded well with clinician ratings (interclass correlation 0.74). This agreement was consistent across four clinical sites. This result demonstrates how such technology can be successfully deployed into existing clinical workflows, with consumer-grade smartphone or tablet devices, adding minimal equipment cost and time.
Collapse
Affiliation(s)
- Gareth Morinan
- Machine Medicine Technologies Ltd., The Leather Market Unit 1.1.1 11/13 Weston Street, London, SE1 3ER UK
| | - Yuriy Dushin
- Machine Medicine Technologies Ltd., The Leather Market Unit 1.1.1 11/13 Weston Street, London, SE1 3ER, UK.
| | - Grzegorz Sarapata
- Machine Medicine Technologies Ltd., The Leather Market Unit 1.1.1 11/13 Weston Street, London, SE1 3ER UK
| | - Samuel Rupprechter
- Machine Medicine Technologies Ltd., The Leather Market Unit 1.1.1 11/13 Weston Street, London, SE1 3ER UK
| | - Yuwei Peng
- Machine Medicine Technologies Ltd., The Leather Market Unit 1.1.1 11/13 Weston Street, London, SE1 3ER UK
| | - Christine Girges
- grid.436283.80000 0004 0612 2631Department of Clinical and Movement Neurosciences, Institute of Neurology, University College London, Queen Square, London, WC1N 3BG UK
| | - Maricel Salazar
- grid.436283.80000 0004 0612 2631Department of Clinical and Movement Neurosciences, Institute of Neurology, University College London, Queen Square, London, WC1N 3BG UK
| | - Catherine Milabo
- grid.436283.80000 0004 0612 2631Department of Clinical and Movement Neurosciences, Institute of Neurology, University College London, Queen Square, London, WC1N 3BG UK
| | - Krista Sibley
- grid.436283.80000 0004 0612 2631Department of Clinical and Movement Neurosciences, Institute of Neurology, University College London, Queen Square, London, WC1N 3BG UK
| | - Thomas Foltynie
- grid.436283.80000 0004 0612 2631Department of Clinical and Movement Neurosciences, Institute of Neurology, University College London, Queen Square, London, WC1N 3BG UK
| | - Ioana Cociasu
- grid.264200.20000 0000 8546 682XNeuroscience Research Centre, Molecular and Clinical Sciences Research Institute, St George’s, University of London, Cranmer Terrace, London, SW17 0RE UK
| | - Lucia Ricciardi
- grid.264200.20000 0000 8546 682XNeuroscience Research Centre, Molecular and Clinical Sciences Research Institute, St George’s, University of London, Cranmer Terrace, London, SW17 0RE UK
| | - Fahd Baig
- grid.264200.20000 0000 8546 682XNeuroscience Research Centre, Molecular and Clinical Sciences Research Institute, St George’s, University of London, Cranmer Terrace, London, SW17 0RE UK
| | - Francesca Morgante
- grid.264200.20000 0000 8546 682XNeuroscience Research Centre, Molecular and Clinical Sciences Research Institute, St George’s, University of London, Cranmer Terrace, London, SW17 0RE UK ,grid.10438.3e0000 0001 2178 8421Department of Clinical and Experimental Medicine, University of Messina, Messina, Italy, Via Consolare Valeria, 98165 Messina, Italy
| | - Louise-Ann Leyland
- grid.436283.80000 0004 0612 2631Dementia Research Center, Institute of Neurology, University College London, Queen Square, London, WC1N 3AR UK
| | - Rimona S. Weil
- grid.436283.80000 0004 0612 2631Dementia Research Center, Institute of Neurology, University College London, Queen Square, London, WC1N 3AR UK
| | - Ro’ee Gilron
- grid.266102.10000 0001 2297 6811The Starr Lab, University of California San Francisco, 513 Parnassus Ave, HSE-823, San Francisco, CA 94143 USA
| | - Jonathan O’Keeffe
- Machine Medicine Technologies Ltd., The Leather Market Unit 1.1.1 11/13 Weston Street, London, SE1 3ER UK
| |
Collapse
|
23
|
Wu Z, Gu H, Hong R, Xing Z, Zhang Z, Peng K, He Y, Xie L, Zhang J, Gao Y, Jin Y, Su X, Zhi H, Guan Q, Pan L, Jin L. Kinect-based objective evaluation of bradykinesia in patients with Parkinson's disease. Digit Health 2023; 9:20552076231176653. [PMID: 37223774 PMCID: PMC10201004 DOI: 10.1177/20552076231176653] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/31/2023] [Accepted: 05/02/2023] [Indexed: 05/25/2023] Open
Abstract
Objective To quantify bradykinesia in Parkinson's disease (PD) with a Kinect depth camera-based motion analysis system and to compare PD and healthy control (HC) subjects. Methods Fifty PD patients and twenty-five HCs were recruited. The Movement Disorder Society-Sponsored Revision of the Unified Parkinson's Disease Rating Scale part III (MDS-UPDRS III) was used to evaluate the motor symptoms of PD. Kinematic features of five bradykinesia-related motor tasks were collected using Kinect depth camera. Then, kinematic features were correlated with the clinical scales and compared between groups. Results Significant correlations were found between kinematic features and clinical scales (P < 0.05). Compared with HCs, PD patients exhibited a significant decrease in the frequency of finger tapping (P < 0.001), hand movement (P < 0.001), hand pronation-supination movements (P = 0.005), and leg agility (P = 0.003). Meanwhile, PD patients had a significant decrease in the speed of hand movements (P = 0.003) and toe tapping (P < 0.001) compared with HCs. Several kinematic features exhibited potential diagnostic value in distinguishing PD from HCs with area under the curve (AUC) ranging from 0.684-0.894 (P < 0.05). Furthermore, the combination of motor tasks exhibited the best diagnostic value with the highest AUC of 0.955 (95% CI = 0.913-0.997, P < 0.001). Conclusion The Kinect-based motion analysis system can be applied to evaluate bradykinesia in PD. Kinematic features can be used to differentiate PD patients from HCs and combining kinematic features from different motor tasks can significantly improve the diagnostic value.
Collapse
Affiliation(s)
- Zhuang Wu
- Department of Neurology and Neurological Rehabilitation, Shanghai Disabled Persons’ Federation Key Laboratory of Intelligent Rehabilitation Assistive Devices and Technologies, Yangzhi Rehabilitation Hospital (Shanghai Sunshine Rehabilitation Center), School of Medicine, Tongji University, Shanghai, China
- Neurotoxin Research Center, Key Laboratory of Spine and Spinal Cord Injury Repair and Regeneration of Ministry of Education, Department of Neurology, Tongji Hospital, School of Medicine, Tongji University, Shanghai, China
| | - Hongkai Gu
- Department of Neurology and Neurological Rehabilitation, Shanghai Disabled Persons’ Federation Key Laboratory of Intelligent Rehabilitation Assistive Devices and Technologies, Yangzhi Rehabilitation Hospital (Shanghai Sunshine Rehabilitation Center), School of Medicine, Tongji University, Shanghai, China
- Neurotoxin Research Center, Key Laboratory of Spine and Spinal Cord Injury Repair and Regeneration of Ministry of Education, Department of Neurology, Tongji Hospital, School of Medicine, Tongji University, Shanghai, China
| | - Ronghua Hong
- Department of Neurology and Neurological Rehabilitation, Shanghai Disabled Persons’ Federation Key Laboratory of Intelligent Rehabilitation Assistive Devices and Technologies, Yangzhi Rehabilitation Hospital (Shanghai Sunshine Rehabilitation Center), School of Medicine, Tongji University, Shanghai, China
- Neurotoxin Research Center, Key Laboratory of Spine and Spinal Cord Injury Repair and Regeneration of Ministry of Education, Department of Neurology, Tongji Hospital, School of Medicine, Tongji University, Shanghai, China
| | - Ziwen Xing
- Department of Neurology and Neurological Rehabilitation, Shanghai Disabled Persons’ Federation Key Laboratory of Intelligent Rehabilitation Assistive Devices and Technologies, Yangzhi Rehabilitation Hospital (Shanghai Sunshine Rehabilitation Center), School of Medicine, Tongji University, Shanghai, China
- Neurotoxin Research Center, Key Laboratory of Spine and Spinal Cord Injury Repair and Regeneration of Ministry of Education, Department of Neurology, Tongji Hospital, School of Medicine, Tongji University, Shanghai, China
| | - Zhuoyu Zhang
- Department of Neurology and Neurological Rehabilitation, Shanghai Disabled Persons’ Federation Key Laboratory of Intelligent Rehabilitation Assistive Devices and Technologies, Yangzhi Rehabilitation Hospital (Shanghai Sunshine Rehabilitation Center), School of Medicine, Tongji University, Shanghai, China
- Neurotoxin Research Center, Key Laboratory of Spine and Spinal Cord Injury Repair and Regeneration of Ministry of Education, Department of Neurology, Tongji Hospital, School of Medicine, Tongji University, Shanghai, China
| | - Kangwen Peng
- Department of Neurology and Neurological Rehabilitation, Shanghai Disabled Persons’ Federation Key Laboratory of Intelligent Rehabilitation Assistive Devices and Technologies, Yangzhi Rehabilitation Hospital (Shanghai Sunshine Rehabilitation Center), School of Medicine, Tongji University, Shanghai, China
- Neurotoxin Research Center, Key Laboratory of Spine and Spinal Cord Injury Repair and Regeneration of Ministry of Education, Department of Neurology, Tongji Hospital, School of Medicine, Tongji University, Shanghai, China
| | - Yijing He
- Department of Neurology and Neurological Rehabilitation, Shanghai Disabled Persons’ Federation Key Laboratory of Intelligent Rehabilitation Assistive Devices and Technologies, Yangzhi Rehabilitation Hospital (Shanghai Sunshine Rehabilitation Center), School of Medicine, Tongji University, Shanghai, China
- Neurotoxin Research Center, Key Laboratory of Spine and Spinal Cord Injury Repair and Regeneration of Ministry of Education, Department of Neurology, Tongji Hospital, School of Medicine, Tongji University, Shanghai, China
| | - Ludi Xie
- Department of Neurology and Neurological Rehabilitation, Shanghai Disabled Persons’ Federation Key Laboratory of Intelligent Rehabilitation Assistive Devices and Technologies, Yangzhi Rehabilitation Hospital (Shanghai Sunshine Rehabilitation Center), School of Medicine, Tongji University, Shanghai, China
- Neurotoxin Research Center, Key Laboratory of Spine and Spinal Cord Injury Repair and Regeneration of Ministry of Education, Department of Neurology, Tongji Hospital, School of Medicine, Tongji University, Shanghai, China
| | - Jingxing Zhang
- Neurotoxin Research Center, Key Laboratory of Spine and Spinal Cord Injury Repair and Regeneration of Ministry of Education, Department of Neurology, Tongji Hospital, School of Medicine, Tongji University, Shanghai, China
| | - Yichen Gao
- IFLYTEK Suzhou Research Institute, Suzhou, China
| | - Yue Jin
- IFLYTEK Suzhou Research Institute, Suzhou, China
| | - Xiaoyun Su
- IFLYTEK Suzhou Research Institute, Suzhou, China
| | - Hongping Zhi
- IFLYTEK Suzhou Research Institute, Suzhou, China
| | - Qiang Guan
- Neurotoxin Research Center, Key Laboratory of Spine and Spinal Cord Injury Repair and Regeneration of Ministry of Education, Department of Neurology, Tongji Hospital, School of Medicine, Tongji University, Shanghai, China
| | - Lizhen Pan
- Neurotoxin Research Center, Key Laboratory of Spine and Spinal Cord Injury Repair and Regeneration of Ministry of Education, Department of Neurology, Tongji Hospital, School of Medicine, Tongji University, Shanghai, China
| | - Lingjing Jin
- Department of Neurology and Neurological Rehabilitation, Shanghai Disabled Persons’ Federation Key Laboratory of Intelligent Rehabilitation Assistive Devices and Technologies, Yangzhi Rehabilitation Hospital (Shanghai Sunshine Rehabilitation Center), School of Medicine, Tongji University, Shanghai, China
- Neurotoxin Research Center, Key Laboratory of Spine and Spinal Cord Injury Repair and Regeneration of Ministry of Education, Department of Neurology, Tongji Hospital, School of Medicine, Tongji University, Shanghai, China
- Collaborative Innovation Center for Brain Science, Tongji University, Shanghai, China
| |
Collapse
|
24
|
Smartphone video nystagmography using convolutional neural networks: ConVNG. J Neurol 2022; 270:2518-2530. [PMID: 36422668 PMCID: PMC10129923 DOI: 10.1007/s00415-022-11493-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/08/2022] [Revised: 11/14/2022] [Accepted: 11/16/2022] [Indexed: 11/27/2022]
Abstract
Abstract
Background
Eye movement abnormalities are commonplace in neurological disorders. However, unaided eye movement assessments lack granularity. Although videooculography (VOG) improves diagnostic accuracy, resource intensiveness precludes its broad use. To bridge this care gap, we here validate a framework for smartphone video-based nystagmography capitalizing on recent computer vision advances.
Methods
A convolutional neural network was fine-tuned for pupil tracking using > 550 annotated frames: ConVNG. In a cross-sectional approach, slow-phase velocity of optokinetic nystagmus was calculated in 10 subjects using ConVNG and VOG. Equivalence of accuracy and precision was assessed using the “two one-sample t-test” (TOST) and Bayesian interval-null approaches. ConVNG was systematically compared to OpenFace and MediaPipe as computer vision (CV) benchmarks for gaze estimation.
Results
ConVNG tracking accuracy reached 9–15% of an average pupil diameter. In a fully independent clinical video dataset, ConVNG robustly detected pupil keypoints (median prediction confidence 0.85). SPV measurement accuracy was equivalent to VOG (TOST p < 0.017; Bayes factors (BF) > 24). ConVNG, but not MediaPipe, achieved equivalence to VOG in all SPV calculations. Median precision was 0.30°/s for ConVNG, 0.7°/s for MediaPipe and 0.12°/s for VOG. ConVNG precision was significantly higher than MediaPipe in vertical planes, but both algorithms’ precision was inferior to VOG.
Conclusions
ConVNG enables offline smartphone video nystagmography with an accuracy comparable to VOG and significantly higher precision than MediaPipe, a benchmark computer vision application for gaze estimation. This serves as a blueprint for highly accessible tools with potential to accelerate progress toward precise and personalized Medicine.
Collapse
|
25
|
Itokazu M. Reliability and accuracy of 2D lower limb joint angles during a standing-up motion for markerless motion analysis software using deep learning. MEDICINE IN NOVEL TECHNOLOGY AND DEVICES 2022. [DOI: 10.1016/j.medntd.2022.100188] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/05/2022] Open
|
26
|
Weber RZ, Mulders G, Kaiser J, Tackenberg C, Rust R. Deep learning-based behavioral profiling of rodent stroke recovery. BMC Biol 2022; 20:232. [PMID: 36243716 PMCID: PMC9571460 DOI: 10.1186/s12915-022-01434-9] [Citation(s) in RCA: 14] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/24/2022] [Accepted: 10/05/2022] [Indexed: 11/16/2022] Open
Abstract
BACKGROUND Stroke research heavily relies on rodent behavior when assessing underlying disease mechanisms and treatment efficacy. Although functional motor recovery is considered the primary targeted outcome, tests in rodents are still poorly reproducible and often unsuitable for unraveling the complex behavior after injury. RESULTS Here, we provide a comprehensive 3D gait analysis of mice after focal cerebral ischemia based on the new deep learning-based software (DeepLabCut, DLC) that only requires basic behavioral equipment. We demonstrate a high precision 3D tracking of 10 body parts (including all relevant joints and reference landmarks) in several mouse strains. Building on this rigor motion tracking, a comprehensive post-analysis (with >100 parameters) unveils biologically relevant differences in locomotor profiles after a stroke over a time course of 3 weeks. We further refine the widely used ladder rung test using deep learning and compare its performance to human annotators. The generated DLC-assisted tests were then benchmarked to five widely used conventional behavioral set-ups (neurological scoring, rotarod, ladder rung walk, cylinder test, and single-pellet grasping) regarding sensitivity, accuracy, time use, and costs. CONCLUSIONS We conclude that deep learning-based motion tracking with comprehensive post-analysis provides accurate and sensitive data to describe the complex recovery of rodents following a stroke. The experimental set-up and analysis can also benefit a range of other neurological injuries that affect locomotion.
Collapse
Affiliation(s)
- Rebecca Z Weber
- Institute for Regenerative Medicine (IREM), University of Zurich, Campus Schlieren, Wagistrasse 12, 8952, Schlieren, Switzerland
- Neuroscience Center Zurich, University of Zurich and ETH Zurich, Zurich, Switzerland
| | - Geertje Mulders
- Department of Health Sciences and Technology, ETH Zurich, Zurich, Switzerland
| | - Julia Kaiser
- Burke Neurological Institute, White Plains, NY, USA
| | - Christian Tackenberg
- Institute for Regenerative Medicine (IREM), University of Zurich, Campus Schlieren, Wagistrasse 12, 8952, Schlieren, Switzerland.
- Neuroscience Center Zurich, University of Zurich and ETH Zurich, Zurich, Switzerland.
| | - Ruslan Rust
- Institute for Regenerative Medicine (IREM), University of Zurich, Campus Schlieren, Wagistrasse 12, 8952, Schlieren, Switzerland.
- Neuroscience Center Zurich, University of Zurich and ETH Zurich, Zurich, Switzerland.
| |
Collapse
|
27
|
Alty J, Bai Q, Li R, Lawler K, St George RJ, Hill E, Bindoff A, Garg S, Wang X, Huang G, Zhang K, Rudd KD, Bartlett L, Goldberg LR, Collins JM, Hinder MR, Naismith SL, Hogg DC, King AE, Vickers JC. The TAS Test project: a prospective longitudinal validation of new online motor-cognitive tests to detect preclinical Alzheimer's disease and estimate 5-year risks of cognitive decline and dementia. BMC Neurol 2022; 22:266. [PMID: 35850660 PMCID: PMC9289357 DOI: 10.1186/s12883-022-02772-5] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/18/2022] [Accepted: 06/27/2022] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND The worldwide prevalence of dementia is rapidly rising. Alzheimer's disease (AD), accounts for 70% of cases and has a 10-20-year preclinical period, when brain pathology covertly progresses before cognitive symptoms appear. The 2020 Lancet Commission estimates that 40% of dementia cases could be prevented by modifying lifestyle/medical risk factors. To optimise dementia prevention effectiveness, there is urgent need to identify individuals with preclinical AD for targeted risk reduction. Current preclinical AD tests are too invasive, specialist or costly for population-level assessments. We have developed a new online test, TAS Test, that assesses a range of motor-cognitive functions and has capacity to be delivered at significant scale. TAS Test combines two innovations: using hand movement analysis to detect preclinical AD, and computer-human interface technologies to enable robust 'self-testing' data collection. The aims are to validate TAS Test to [1] identify preclinical AD, and [2] predict risk of cognitive decline and AD dementia. METHODS Aim 1 will be addressed through a cross-sectional study of 500 cognitively healthy older adults, who will complete TAS Test items comprising measures of motor control, processing speed, attention, visuospatial ability, memory and language. TAS Test measures will be compared to a blood-based AD biomarker, phosphorylated tau 181 (p-tau181). Aim 2 will be addressed through a 5-year prospective cohort study of 10,000 older adults. Participants will complete TAS Test annually and subtests of the Cambridge Neuropsychological Test Battery (CANTAB) biennially. 300 participants will undergo in-person clinical assessments. We will use machine learning of motor-cognitive performance on TAS Test to develop an algorithm that classifies preclinical AD risk (p-tau181-defined) and determine the precision to prospectively estimate 5-year risks of cognitive decline and AD. DISCUSSION This study will establish the precision of TAS Test to identify preclinical AD and estimate risk of cognitive decline and AD. If accurate, TAS Test will provide a low-cost, accessible enrichment strategy to pre-screen individuals for their likelihood of AD pathology prior to more expensive tests such as blood or imaging biomarkers. This would have wide applications in public health initiatives and clinical trials. TRIAL REGISTRATION ClinicalTrials.gov Identifier: NCT05194787 , 18 January 2022. Retrospectively registered.
Collapse
Affiliation(s)
- Jane Alty
- Wicking Dementia Research and Education Centre, University of Tasmania, Hobart, Australia. .,School of Medicine, University of Tasmania, Hobart, Australia. .,Royal Hobart Hospital, Hobart, Tasmania, Australia.
| | - Quan Bai
- School of Information and Communication Technologies, University of Tasmania, Hobart, Australia
| | - Renjie Li
- School of Information and Communication Technologies, University of Tasmania, Hobart, Australia
| | - Katherine Lawler
- Wicking Dementia Research and Education Centre, University of Tasmania, Hobart, Australia.,Royal Hobart Hospital, Hobart, Tasmania, Australia
| | - Rebecca J St George
- Wicking Dementia Research and Education Centre, University of Tasmania, Hobart, Australia.,School of Psychological Sciences, University of Tasmania, Hobart, Australia
| | - Edward Hill
- Wicking Dementia Research and Education Centre, University of Tasmania, Hobart, Australia
| | - Aidan Bindoff
- Wicking Dementia Research and Education Centre, University of Tasmania, Hobart, Australia
| | - Saurabh Garg
- School of Information and Communication Technologies, University of Tasmania, Hobart, Australia
| | - Xinyi Wang
- School of Information and Communication Technologies, University of Tasmania, Hobart, Australia
| | - Guan Huang
- School of Information and Communication Technologies, University of Tasmania, Hobart, Australia
| | - Kaining Zhang
- School of Information and Communication Technologies, University of Tasmania, Hobart, Australia
| | - Kaylee D Rudd
- Wicking Dementia Research and Education Centre, University of Tasmania, Hobart, Australia
| | - Larissa Bartlett
- Wicking Dementia Research and Education Centre, University of Tasmania, Hobart, Australia
| | - Lynette R Goldberg
- Wicking Dementia Research and Education Centre, University of Tasmania, Hobart, Australia
| | - Jessica M Collins
- Wicking Dementia Research and Education Centre, University of Tasmania, Hobart, Australia
| | - Mark R Hinder
- School of Psychological Sciences, University of Tasmania, Hobart, Australia
| | - Sharon L Naismith
- Healthy Brain Ageing Program, University of Sydney, Sydney, Australia
| | - David C Hogg
- School of Computing, University of Leeds, Leeds, UK
| | - Anna E King
- Wicking Dementia Research and Education Centre, University of Tasmania, Hobart, Australia
| | - James C Vickers
- Wicking Dementia Research and Education Centre, University of Tasmania, Hobart, Australia
| |
Collapse
|
28
|
Kothare H, Roesler O, Burke W, Neumann M, Liscombe J, Exner A, Snyder S, Cornish A, Habberstad D, Pautler D, Suendermann-Oeft D, Huber J, Ramanarayanan V. Speech, Facial and Fine Motor Features for Conversation-Based Remote Assessment and Monitoring of Parkinson's Disease. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2022; 2022:3464-3467. [PMID: 36086652 DOI: 10.1109/embc48229.2022.9871375] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
We present a cloud-based multimodal dialogue platform for the remote assessment and monitoring of speech, facial and fine motor function in Parkinson's Disease (PD) at scale, along with a preliminary investigation of the efficacy of the various metrics automatically extracted by the platform. 22 healthy controls and 38 people with Parkinson's Disease (pPD) were instructed to complete four interactive sessions, spaced a week apart, on the platform. Each session involved a battery of tasks designed to elicit speech, facial movements and finger movements. We find that speech, facial kinematic and finger movement dexterity metrics show statistically significant differences between controls and pPD. We further investigate the sensitivity, specificity, reliability and generalisability of these metrics. Our results offer encouraging evidence for the utility of automatically-extracted audiovisual analytics in remote mon-itoring of PD and other movement disorders.
Collapse
|
29
|
Li R, St George RJ, Wang X, Lawler K, Hill E, Garg S, Williams S, Relton S, Hogg D, Bai Q, Alty J. Moving towards intelligent telemedicine: Computer vision measurement of human movement. Comput Biol Med 2022; 147:105776. [PMID: 35780600 PMCID: PMC9428734 DOI: 10.1016/j.compbiomed.2022.105776] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/01/2022] [Revised: 05/31/2022] [Accepted: 06/19/2022] [Indexed: 11/29/2022]
Abstract
Background: Telemedicine video consultations are rapidly increasing globally, accelerated by the COVID-19 pandemic. This presents opportunities to use computer vision technologies to augment clinician visual judgement because video cameras are so ubiquitous in personal devices and new techniques, such as DeepLabCut (DLC) can precisely measure human movement from smartphone videos. However, the accuracy of DLC to track human movements in videos obtained from laptop cameras, which have a much lower FPS, has never been investigated; this is a critical gap because patients use laptops for most telemedicine consultations. Objectives: To determine the validity and reliability of DLC applied to laptop videos to measure finger tapping, a validated test of human movement. Method: Sixteen adults completed finger-tapping tests at 0.5 Hz, 1 Hz, 2 Hz, 3 Hz and at maximal speed. Hand movements were recorded simultaneously by a laptop camera at 30 frames per second (FPS) and by Optotrak, a 3D motion analysis system at 250 FPS. Eight DLC neural network architectures (ResNet50, ResNet101, ResNet152, MobileNetV1, MobileNetV2, EfficientNetB0, EfficientNetB3, EfficientNetB6) were applied to the laptop video and extracted movement features were compared to the ground truth Optotrak motion tracking. Results: Over 96% (529/552) of DLC measures were within +/−0.5 Hz of the Optotrak measures. At tapping frequencies >4 Hz, there was progressive decline in accuracy, attributed to motion blur associated with the laptop camera’s low FPS. Computer vision methods hold potential for moving us towards intelligent telemedicine by providing human movement analysis during consultations. However, further developments are required to accurately measure the fastest movements.
Collapse
Affiliation(s)
- Renjie Li
- Discipline of Information and Communication Technology, Wicking Dementia Research and Education Centre, University of Tasmania, Australia.
| | - Rebecca J St George
- Sensorimotor Neuroscience and Aging Group, School of Psychological Sciences, University of Tasmania, Australia.
| | - Xinyi Wang
- Discipline of Information and Communication Technology, Wicking Dementia Research and Education Centre, University of Tasmania, Australia.
| | - Katherine Lawler
- Wicking Dementia Research and Education Centre, University of Tasmania, Australia.
| | - Edward Hill
- Wicking Dementia Research and Education Centre, University of Tasmania, Australia.
| | - Saurabh Garg
- Discipline of Information and Communication Technology, University of Tasmania, Australia.
| | | | - Samuel Relton
- School of Medicine, University of Leeds, United Kingdom.
| | - David Hogg
- School of Computing, University of Leeds, United Kingdom.
| | - Quan Bai
- Discipline of Information and Communication Technology, University of Tasmania, Australia.
| | - Jane Alty
- Wicking Dementia Research and Education Centre, University of Tasmania, Australia.
| |
Collapse
|
30
|
Hoang TH, Zehni M, Xu H, Heintz G, Zallek C, Do MN. Towards a Comprehensive Solution for a Vision-based Digitized Neurological Examination. IEEE J Biomed Health Inform 2022; 26:4020-4031. [PMID: 35439148 DOI: 10.1109/jbhi.2022.3167927] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
The ability to use digitally recorded and quantified neurological exam information is important to help healthcare systems deliver better care, in-person and via telehealth, as they compensate for a growing shortage of neurologists. Current neurological digital biomarker pipelines, however, are narrowed down to a specific neurological exam component or applied for assessing specific conditions. In this paper, we propose an accessible vision-based exam and documentation solution called Digitized Neurological Examination (DNE) to expand exam biomarker recording options and clinical applications using a smartphone/tablet. Through our DNE software, healthcare providers in clinical settings and people at home are enabled to video capture an examination while performing instructed neurological tests, including finger tapping, finger to finger, forearm roll, and stand-up and walk. Our modular design of the DNE software supports integrations of additional tests. The DNE extracts from the recorded examinations the 2D/3D human-body pose and quantifies kinematic and spatio-temporal features. The features are clinically relevant and allow clinicians to document and observe the quantified movements and the changes of these metrics over time. A web server and a user interface for recordings viewing and feature visualizations are available. DNE was evaluated on a collected dataset of 21 subjects containing normal and simulated-impaired movements. The overall accuracy of DNE is demonstrated by classifying the recorded movements using various machine learning models. Our tests show an accuracy beyond 90% for upper-limb tests and 80% for the stand-up and walk tests.
Collapse
|
31
|
Guo Z, Zeng W, Yu T, Xu Y, Xiao Y, Cao X, Cao Z. Vision-based Finger Tapping Test in Patients with Parkinson's Disease via Spatial-temporal 3D Hand Pose Estimation. IEEE J Biomed Health Inform 2022; 26:3848-3859. [PMID: 35349459 DOI: 10.1109/jbhi.2022.3162386] [Citation(s) in RCA: 18] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
Finger tapping test is crucial for diagnosing Parkinson's Disease (PD), but manual visual evaluations can result in score discrepancy due to clinicians' subjectivity. Moreover, applying wearable sensors requires making physical contact and may hinder PD patient's raw movement patterns. Accordingly, a novel computer-vision approach is proposed using depth camera and spatial-temporal 3D hand pose estimation to capture and evaluate PD patients' 3D hand movement. Within this approach, a temporal encoding module is leveraged to extend A2J's deep learning framework to counter the pose jittering problem, and a pose refinement process is utilized to alleviate dependency on massive data. Additionally, the first vision-based 3D PD hand dataset of 112 hand samples from 48 PD patients and 11 control subjects is constructed, fully annotated by qualified physicians under clinical settings. Testing on this real-world data, this new model achieves 81.2% classification accuracy, even surpassing that of individual clinicians in comparison, fully demonstrating this proposition's effectiveness. The demo video can be ac-cessed at https://github.com/ZhilinGuo/ST-A2J.
Collapse
|
32
|
Nunes AS, Kozhemiako N, Stephen CD, Schmahmann JD, Khan S, Gupta AS. Automatic Classification and Severity Estimation of Ataxia From Finger Tapping Videos. Front Neurol 2022; 12:795258. [PMID: 35295715 PMCID: PMC8919801 DOI: 10.3389/fneur.2021.795258] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/14/2021] [Accepted: 12/23/2021] [Indexed: 02/06/2023] Open
Abstract
Digital assessments enable objective measurements of ataxia severity and provide informative features that expand upon the information obtained during a clinical examination. In this study, we demonstrate the feasibility of using finger tapping videos to distinguish participants with Ataxia (N = 169) from participants with parkinsonism (N = 78) and from controls (N = 58), and predict their upper extremity and overall disease severity. Features were extracted from the time series representing the distance between the index and thumb and its derivatives. Classification models in ataxia archived areas under the receiver-operating curve of around 0.91, and regression models estimating disease severity obtained correlation coefficients around r = 0.64. Classification and prediction model coefficients were examined and they not only were in accordance, but were in line with clinical observations of ataxia phenotypes where rate and rhythm are altered during upper extremity motor movement.
Collapse
Affiliation(s)
- Adonay S. Nunes
- Department of Neurology, Massachusetts General Hospital and Harvard Medical School, Boston, MA, United States
| | - Nataliia Kozhemiako
- Department of Psychiatry, Brigham and Women's Hospital and Harvard Medical School, Boston, MA, United States
| | - Christopher D. Stephen
- Department of Neurology, Massachusetts General Hospital and Harvard Medical School, Boston, MA, United States,Ataxia Center, Department of Neurology, Massachusetts General Hospital and Harvard Medical School, Boston, MA, United States,Movement Disorders Unit, Department of Neurology, Massachusetts General Hospital and Harvard Medical School, Boston, MA, United States
| | - Jeremy D. Schmahmann
- Department of Neurology, Massachusetts General Hospital and Harvard Medical School, Boston, MA, United States,Ataxia Center, Department of Neurology, Massachusetts General Hospital and Harvard Medical School, Boston, MA, United States
| | - Sheraz Khan
- Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital and Harvard Medical School, Boston, MA, United States,Department of Radiology, Massachusetts General Hospital and Harvard Medical School, Boston, MA, United States
| | - Anoopum S. Gupta
- Department of Neurology, Massachusetts General Hospital and Harvard Medical School, Boston, MA, United States,Ataxia Center, Department of Neurology, Massachusetts General Hospital and Harvard Medical School, Boston, MA, United States,Movement Disorders Unit, Department of Neurology, Massachusetts General Hospital and Harvard Medical School, Boston, MA, United States,*Correspondence: Anoopum S. Gupta
| |
Collapse
|
33
|
Fröhlich H, Bontridder N, Petrovska-Delacréta D, Glaab E, Kluge F, Yacoubi ME, Marín Valero M, Corvol JC, Eskofier B, Van Gyseghem JM, Lehericy S, Winkler J, Klucken J. Leveraging the Potential of Digital Technology for Better Individualized Treatment of Parkinson's Disease. Front Neurol 2022; 13:788427. [PMID: 35295840 PMCID: PMC8918525 DOI: 10.3389/fneur.2022.788427] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/02/2021] [Accepted: 01/31/2022] [Indexed: 12/18/2022] Open
Abstract
Recent years have witnessed a strongly increasing interest in digital technology within medicine (sensor devices, specific smartphone apps) and specifically also neurology. Quantitative measures derived from digital technology could provide Digital Biomarkers (DMs) enabling a quantitative and continuous monitoring of disease symptoms, also outside clinics. This includes the possibility to continuously and sensitively monitor the response to treatment, hence opening the opportunity to adapt medication pathways quickly. In addition, DMs may in the future allow early diagnosis, stratification of patient subgroups and prediction of clinical outcomes. Thus, DMs could complement or in certain cases even replace classical examiner-based outcome measures and molecular biomarkers measured in cerebral spinal fluid, blood, urine, saliva, or other body liquids. Altogether, DMs could play a prominent role in the emerging field of precision medicine. However, realizing this vision requires dedicated research. First, advanced data analytical methods need to be developed and applied, which extract candidate DMs from raw signals. Second, these candidate DMs need to be validated by (a) showing their correlation to established clinical outcome measures, and (b) demonstrating their diagnostic and/or prognostic value compared to established biomarkers. These points again require the use of advanced data analytical methods, including machine learning. In addition, the arising ethical, legal and social questions associated with the collection and processing of sensitive patient data and the use of machine learning methods to analyze these data for better individualized treatment of the disease, must be considered thoroughly. Using Parkinson's Disease (PD) as a prime example of a complex multifactorial disorder, the purpose of this article is to critically review the current state of research regarding the use of DMs, discuss open challenges and highlight emerging new directions.
Collapse
Affiliation(s)
- Holger Fröhlich
- Department of Bioinformatics, Fraunhofer Institute for Algorithms and Scientific Computing, Sankt Augustin, Germany
- Bonn-Aachen International Center for IT (b-it), University of Bonn, Bonn, Germany
| | - Noémi Bontridder
- Centre de Recherches Information, Droit et Societe, University of Namur, Namur, Belgium
| | | | - Enrico Glaab
- Luxembourg Center for Systems Medicine, University of Luxembourg, Esch, Luxembourg
| | - Felix Kluge
- Department of Artificial Intelligence in Biomedical Engineering, University of Erlangen Nuremberg, Erlangen, Germany
| | | | | | | | - Bjoern Eskofier
- Department of Artificial Intelligence in Biomedical Engineering, University of Erlangen Nuremberg, Erlangen, Germany
| | | | | | - Jürgen Winkler
- Department of Neurology, University Hospital Erlangen, Erlangen, Germany
| | - Jochen Klucken
- Luxembourg Center for Systems Medicine, University of Luxembourg, Esch, Luxembourg
| |
Collapse
|
34
|
Monje MHG, Domínguez S, Vera-Olmos J, Antonini A, Mestre TA, Malpica N, Sánchez-Ferro Á. Remote Evaluation of Parkinson's Disease Using a Conventional Webcam and Artificial Intelligence. Front Neurol 2022; 12:742654. [PMID: 35002915 PMCID: PMC8733479 DOI: 10.3389/fneur.2021.742654] [Citation(s) in RCA: 11] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/16/2021] [Accepted: 10/18/2021] [Indexed: 11/25/2022] Open
Abstract
Objective: This study aimed to prove the concept of a new optical video-based system to measure Parkinson's disease (PD) remotely using an accessible standard webcam. Methods: We consecutively enrolled a cohort of 42 patients with PD and healthy subjects (HSs). The participants were recorded performing MDS-UPDRS III bradykinesia upper limb tasks with a computer webcam. The video frames were processed using the artificial intelligence algorithms tracking the movements of the hands. The video extracted features were correlated with clinical rating using the Movement Disorder Society revision of the Unified Parkinson's Disease Rating Scale and inertial measurement units (IMUs). The developed classifiers were validated on an independent dataset. Results: We found significant differences in the motor performance of the patients with PD and HSs in all the bradykinesia upper limb motor tasks. The best performing classifiers were unilateral finger tapping and hand movement speed. The model correlated both with the IMUs for quantitative assessment of motor function and the clinical scales, hence demonstrating concurrent validity with the existing methods. Conclusions: We present here the proof-of-concept of a novel webcam-based technology to remotely detect the parkinsonian features using artificial intelligence. This method has preliminarily achieved a very high diagnostic accuracy and could be easily expanded to other disease manifestations to support PD management.
Collapse
Affiliation(s)
- Mariana H G Monje
- HM CINAC (Centro Integral de Neurociencias Abarca Campal), Hospital Universitario HM Puerta del Sur, HM Hospitales, Madrid, Spain.,Department of Neurology, Northwestern University Feinberg School of Medicine, Chicago, IL, United States
| | - Sergio Domínguez
- LAIMBIO, Laboratorio de Análisis de Imagen Médica y Biometría, Universidad Rey Juan Carlos, Madrid, Spain
| | - Javier Vera-Olmos
- LAIMBIO, Laboratorio de Análisis de Imagen Médica y Biometría, Universidad Rey Juan Carlos, Madrid, Spain
| | - Angelo Antonini
- Parkinson and Movement Disorders Unit, Department of Neurosciences (DNS), Padova University, Padova, Italy
| | - Tiago A Mestre
- Division of Neurology, Department of Medicine, Parkinson's Disease and Movement Disorders Centre, The Ottawa Hospital Research Institute, The University of Ottawa Brain Research Institute, Ottawa, ON, Canada
| | - Norberto Malpica
- LAIMBIO, Laboratorio de Análisis de Imagen Médica y Biometría, Universidad Rey Juan Carlos, Madrid, Spain
| | - Álvaro Sánchez-Ferro
- HM CINAC (Centro Integral de Neurociencias Abarca Campal), Hospital Universitario HM Puerta del Sur, HM Hospitales, Madrid, Spain.,Movement Disorders Unit, Neurology Department, Hospital Universitario 12 de Octubre, Madrid, Spain
| |
Collapse
|
35
|
Developing and assessing a new web-based tapping test for measuring distal movement in Parkinson's disease: a Distal Finger Tapping test. Sci Rep 2022; 12:386. [PMID: 35013372 PMCID: PMC8748736 DOI: 10.1038/s41598-021-03563-7] [Citation(s) in RCA: 12] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/20/2021] [Accepted: 11/30/2021] [Indexed: 11/08/2022] Open
Abstract
Disability in Parkinson's disease (PD) is measured by standardised scales including the MDS-UPDRS, which are subject to high inter and intra-rater variability and fail to capture subtle motor impairment. The BRadykinesia Akinesia INcoordination (BRAIN) test is a validated keyboard tapping test, evaluating proximal upper-limb motor impairment. Here, a new Distal Finger Tapping (DFT) test was developed to assess distal upper-limb function. Kinetic parameters of the test include kinesia score (KS20, key taps over 20 s), akinesia time (AT20, mean dwell-time on each key) and incoordination score (IS20, variance of travelling time between key taps). To develop and evaluate a new keyboard-tapping test for objective and remote distal motor function in PD patients. The DFT and BRAIN tests were assessed in 55 PD patients and 65 controls. Test scores were compared between groups and correlated with the MDS-UPDRS-III finger tapping sub-scores. Nine additional PD patients were recruited for monitoring motor fluctuations. All three parameters discriminated effectively between PD patients and controls, with KS20 performing best, yielding 79% sensitivity for 85% specificity; area under the receiver operating characteristic curve (AUC) = 0.90. A combination of DFT and BRAIN tests improved discrimination (AUC = 0.95). Among three parameters, KS20 showed a moderate correlation with the MDS-UPDRS finger-tapping sub-score (Pearson's r = - 0.40, p = 0.002). Further, the DFT test detected subtle changes in motor fluctuation states which were not reflected clearly by the MDS-UPDRS-III finger tapping sub-scores. The DFT test is an online tool for assessing distal movements in PD, with future scope for longitudinal monitoring of motor complications.
Collapse
|
36
|
Sibley K, Girges C, Candelario J, Milabo C, Salazar M, Esperida JO, Dushin Y, Limousin P, Foltynie T. An Evaluation of KELVIN, an Artificial Intelligence Platform, as an Objective Assessment of the MDS UPDRS Part III. JOURNAL OF PARKINSON'S DISEASE 2022; 12:2223-2233. [PMID: 36155530 DOI: 10.3233/jpd-223493] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/16/2023]
Abstract
BACKGROUND Parkinson's disease severity is typically measured using the Movement Disorder Society Unified Parkinson's disease rating scale (MDS-UPDRS). While training for this scale exists, users may vary in how they score a patient with the consequence of intra-rater and inter-rater variability. OBJECTIVE In this study we explored the consistency of an artificial intelligence platform compared with traditional clinical scoring in the assessment of motor severity in PD. METHODS Twenty-two PD patients underwent simultaneous MDS-UPDRS scoring by two experienced MDS-UPDRS raters and the two sets of accompanying video footage were also scored by an artificial intelligence video analysis platform known as KELVIN. RESULTS KELVIN was able to produce a summary score for 7 MDS-UPDRS part 3 items with good inter-rater reliability (Intraclass Correlation Coefficient (ICC) 0.80 in the OFF-medication state, ICC 0.73 in the ON-medication state). Clinician scores had exceptionally high levels of inter-rater reliability in both the OFF (0.99) and ON (0.94) medication conditions (possibly reflecting the highly experienced team). There was an ICC of 0.84 in the OFF-medication state and 0.31 in the ON-medication state between the mean Clinician and mean Kelvin scores for the equivalent 7 motor items, possibly due to dyskinesia impacting on the KELVIN scores. CONCLUSION We conclude that KELVIN may prove useful in the capture and scoring of multiple items of MDS-UPDRS part 3 with levels of consistency not far short of that achieved by experienced MDS-UPDRS clinical raters, and is worthy of further investigation.
Collapse
Affiliation(s)
- Krista Sibley
- Department of Clinical & Movement Neurosciences, UCL Institute of Neurology, National Hospital for Neurology and Neurosurgery, London, UK
| | - Christine Girges
- Department of Clinical & Movement Neurosciences, UCL Institute of Neurology, National Hospital for Neurology and Neurosurgery, London, UK
| | - Joseph Candelario
- Department of Clinical & Movement Neurosciences, UCL Institute of Neurology, National Hospital for Neurology and Neurosurgery, London, UK
| | - Catherine Milabo
- Department of Clinical & Movement Neurosciences, UCL Institute of Neurology, National Hospital for Neurology and Neurosurgery, London, UK
| | - Maricel Salazar
- Department of Clinical & Movement Neurosciences, UCL Institute of Neurology, National Hospital for Neurology and Neurosurgery, London, UK
| | - John Onil Esperida
- Department of Clinical & Movement Neurosciences, UCL Institute of Neurology, National Hospital for Neurology and Neurosurgery, London, UK
| | | | - Patricia Limousin
- Department of Clinical & Movement Neurosciences, UCL Institute of Neurology, National Hospital for Neurology and Neurosurgery, London, UK
| | - Thomas Foltynie
- Department of Clinical & Movement Neurosciences, UCL Institute of Neurology, National Hospital for Neurology and Neurosurgery, London, UK
| |
Collapse
|
37
|
Cornman HL, Stenum J, Roemmich RT. Video-based quantification of human movement frequency using pose estimation: A pilot study. PLoS One 2021; 16:e0261450. [PMID: 34929012 PMCID: PMC8687570 DOI: 10.1371/journal.pone.0261450] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/02/2021] [Accepted: 12/03/2021] [Indexed: 11/18/2022] Open
Abstract
Assessment of repetitive movements (e.g., finger tapping) is a hallmark of motor examinations in several neurologic populations. These assessments are traditionally performed by a human rater via visual inspection; however, advances in computer vision offer potential for remote, quantitative assessment using simple video recordings. Here, we evaluated a pose estimation approach for measurement of human movement frequency from smartphone videos. Ten healthy young participants provided videos of themselves performing five repetitive movement tasks (finger tapping, hand open/close, hand pronation/supination, toe tapping, leg agility) at four target frequencies (1–4 Hz). We assessed the ability of a workflow that incorporated OpenPose (a freely available whole-body pose estimation algorithm) to estimate movement frequencies by comparing against manual frame-by-frame (i.e., ground-truth) measurements for all tasks and target frequencies using repeated measures ANOVA, Pearson’s correlations, and intraclass correlations. Our workflow produced largely accurate estimates of movement frequencies; only the hand open/close task showed a significant difference in the frequencies estimated by pose estimation and manual measurement (while statistically significant, these differences were small in magnitude). All other tasks and frequencies showed no significant differences between pose estimation and manual measurement. Pose estimation-based detections of individual events (e.g., finger taps, hand closures) showed strong correlations (all r>0.99) with manual detections for all tasks and frequencies. In summary, our pose estimation-based workflow accurately tracked repetitive movements in healthy adults across a range of tasks and movement frequencies. Future work will test this approach as a fast, quantitative, video-based approach to assessment of repetitive movements in clinical populations.
Collapse
Affiliation(s)
- Hannah L. Cornman
- Center for Movement Studies, Kennedy Krieger Institute, Baltimore, MD, United States of America
- Dept of Physical Medicine and Rehabilitation, Johns Hopkins University School of Medicine, Baltimore, MD, United States of America
- University of Maryland School of Medicine, Baltimore, MD, United States of America
| | - Jan Stenum
- Center for Movement Studies, Kennedy Krieger Institute, Baltimore, MD, United States of America
- Dept of Physical Medicine and Rehabilitation, Johns Hopkins University School of Medicine, Baltimore, MD, United States of America
| | - Ryan T. Roemmich
- Center for Movement Studies, Kennedy Krieger Institute, Baltimore, MD, United States of America
- Dept of Physical Medicine and Rehabilitation, Johns Hopkins University School of Medicine, Baltimore, MD, United States of America
- * E-mail:
| |
Collapse
|
38
|
Williams S, Fang H, Relton SD, Wong DC, Alam T, Alty JE. Accuracy of Smartphone Video for Contactless Measurement of Hand Tremor Frequency. Mov Disord Clin Pract 2021; 8:69-75. [PMID: 34853806 DOI: 10.1002/mdc3.13119] [Citation(s) in RCA: 19] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/24/2020] [Revised: 09/14/2020] [Accepted: 10/20/2020] [Indexed: 11/05/2022] Open
Abstract
Background Computer vision can measure movement from video without the time and access limitations of hospital accelerometry/electromyography or the requirement to hold or strap a smartphone accelerometer. Objective To compare computer vision measurement of hand tremor frequency from smartphone video with a gold standard measure accelerometer. Methods A total of 37 smartphone videos of hands, at rest and in posture, were recorded from 15 participants with tremor diagnoses (9 Parkinson's disease, 5 essential tremor, 1 functional tremor). Video pixel movement was measured using the computing technique of optical flow, with contemporaneous accelerometer recording. Fast Fourier transform and Bland-Altman analysis were applied. Tremor amplitude was scored by 2 clinicians. Results Bland-Altman analysis of dominant tremor frequency from smartphone video compared with accelerometer showed excellent agreement: 95% limits of agreement -0.38 Hz to +0.35 Hz. In 36 of 37 videos (97%), there was <0.5 Hz difference between computer vision and accelerometer measurement. There was no significant correlation between the level of agreement and tremor amplitude. Conclusion The study suggests a potential new, contactless point-and-press measure of tremor frequency within standard clinical settings, research studies, or telemedicine.
Collapse
Affiliation(s)
- Stefan Williams
- Leeds Institute of Health Science, University of Leeds Leeds UK.,Department of Neurology Leeds Teaching Hospitals National Health Service (NHS) Trust Leeds UK
| | - Hui Fang
- Department of Computer Science Loughborough University Loughborough UK
| | - Samuel D Relton
- Leeds Institute of Health Science, University of Leeds Leeds UK
| | - David C Wong
- Division of Informatics, Imaging and Data Science University of Manchester Manchester UK
| | - Taimour Alam
- Department of Neurology Leeds Teaching Hospitals National Health Service (NHS) Trust Leeds UK
| | - Jane E Alty
- Department of Neurology Leeds Teaching Hospitals National Health Service (NHS) Trust Leeds UK.,Wicking Dementia Research and Education Centre University of Tasmania Hobart Tasmania Australia
| |
Collapse
|
39
|
Garcia-Agundez A, Eickhoff C. Towards Objective Quantification of Hand Tremors and Bradykinesia Using Contactless Sensors: A Systematic Review. Front Aging Neurosci 2021; 13:716102. [PMID: 34759810 PMCID: PMC8572888 DOI: 10.3389/fnagi.2021.716102] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/28/2021] [Accepted: 09/27/2021] [Indexed: 11/26/2022] Open
Abstract
Assessing the progression of movement disorders such as Parkinson's Disease (PD) is key in adjusting therapeutic interventions. However, current methods are still based on subjective factors such as visual observation, resulting in significant inter-rater variability on clinical scales such as UPDRS. Recent studies show the potential of sensor-based methods to address this limitation. The goal of this systematic review is to provide an up-to-date analysis of contactless sensor-based methods to estimate hand dexterity UPDRS scores in PD patients. Two hundred and twenty-four abstracts were screened and nine articles selected for analysis. Evidence obtained in a cumulative cohort of n = 187 patients and 1, 385 samples indicates that contactless sensors, particularly the Leap Motion Controller (LMC), can be used to assess UPDRS hand motor tasks 3.4, 3.5, 3.6, 3.15, and 3.17, although accuracy varies. Early evidence shows that sensor-based methods have clinical potential and might, after refinement, complement, or serve as a support to subjective assessment procedures. Given the nature of UPDRS assessment, future studies should observe whether LMC classification error falls within inter-rater variability for clinician-measured UPDRS scores to validate its clinical utility. Conversely, variables relevant to LMC classification such as power spectral densities or movement opening and closing speeds could set the basis for the design of more objective expert systems to assess hand dexterity in PD.
Collapse
Affiliation(s)
- Augusto Garcia-Agundez
- AI Lab, Brown Center for Biomedical Informatics, Brown University, Providence, RI, United States
| | - Carsten Eickhoff
- AI Lab, Brown Center for Biomedical Informatics, Brown University, Providence, RI, United States
| |
Collapse
|
40
|
Solby H, Radovanovic M, Sommerville JA. A New Look at Infant Problem-Solving: Using DeepLabCut to Investigate Exploratory Problem-Solving Approaches. Front Psychol 2021; 12:705108. [PMID: 34819894 PMCID: PMC8606407 DOI: 10.3389/fpsyg.2021.705108] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/04/2021] [Accepted: 10/18/2021] [Indexed: 12/22/2022] Open
Abstract
When confronted with novel problems, problem-solvers must decide whether to copy a modeled solution or to explore their own unique solutions. While past work has established that infants can learn to solve problems both through their own exploration and through imitation, little work has explored the factors that influence which of these approaches infants select to solve a given problem. Moreover, past work has treated imitation and exploration as qualitatively distinct, although these two possibilities may exist along a continuum. Here, we apply a program novel to developmental psychology (DeepLabCut) to archival data (Lucca et al., 2020) to investigate the influence of the effort and success of an adult's modeled solution, and infants' firsthand experience with failure, on infants' imitative versus exploratory problem-solving approaches. Our results reveal that tendencies toward exploration are relatively immune to the information from the adult model, but that exploration generally increased in response to firsthand experience with failure. In addition, we found that increases in maximum force and decreases in trying time were associated with greater exploration, and that exploration subsequently predicted problem-solving success on a new iteration of the task. Thus, our results demonstrate that infants increase exploration in response to failure and that exploration may operate in a larger motivational framework with force, trying time, and expectations of task success.
Collapse
|
41
|
Geelen JE, Branco MP, Ramsey NF, van der Helm FCT, Mugge W, Schouten AC. MarkerLess Motion Capture: ML-MoCap, a low-cost modular multi-camera setup. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2021; 2021:4859-4862. [PMID: 34892297 DOI: 10.1109/embc46164.2021.9629749] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
Motion capture systems are extensively used to track human movement to study healthy and pathological movements, allowing for objective diagnosis and effective therapy of conditions that affect our motor system. Current motion capture systems typically require marker placements which is cumbersome and can lead to contrived movements.Here, we describe and evaluate our developed markerless and modular multi-camera motion capture system to record human movements in 3D. The system consists of several interconnected single-board microcomputers, each coupled to a camera (i.e., the camera modules), and one additional microcomputer, which acts as the controller. The system allows for integration with upcoming machine-learning techniques, such as DeepLabCut and AniPose. These tools convert the video frames into virtual marker trajectories and provide input for further biomechanical analysis.The system obtains a frame rate of 40 Hz with a sub-millisecond synchronization between the camera modules. We evaluated the system by recording index finger movement using six camera modules. The recordings were converted via trajectories of the bony segments into finger joint angles. The retrieved finger joint angles were compared to a marker-based system resulting in a root-mean-square error of 7.5 degrees difference for a full range metacarpophalangeal joint motion.Our system allows for out-of-the-lab motion capture studies while eliminating the need for reflective markers. The setup is modular by design, enabling various configurations for both coarse and fine movement studies, allowing for machine learning integration to automatically label the data. Although we compared our system for a small movement, this method can also be extended to full-body experiments in larger volumes.
Collapse
|
42
|
Luiz LMD, Marques IA, Folador JP, Andrade AO. Intra and inter-rater remote assessment of bradykinesia in Parkinson's disease. Neurologia 2021:S0213-4853(21)00130-4. [PMID: 34538673 DOI: 10.1016/j.nrl.2021.08.005] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/07/2021] [Revised: 08/02/2021] [Accepted: 08/04/2021] [Indexed: 12/15/2022] Open
Abstract
INTRODUCTION Reliable assessment of individuals with Parkinson's disease (PD) is essential for providing adequate treatment. Clinical assessment is a complex and time-consuming task, especially for bradykinesia, since its evaluation can be influenced by the degree of experience of the examiner, patient collaboration and individual bias. Improvement of the clinical evaluation can be obtained by considering assessments from several professionals. However, this is only true when inter and intra-rater agreement are high. Recently, the Movement Disorder Society highlighted, during the COVID-19 pandemic, the need to develop and validate technologies for remote assessment of the motor status of people with PD. Thus, this study introduces an objective strategy for the remote evaluation of bradykinesia using multi-specialist analysis. METHODS Twelve volunteers with PD participated and these were asked to execute finger tapping, hand opening/closing and pronation/supination movements. Each task was recorded and rated by fourteen PD health experts for each patient. The scores were assessed on an individual basis. Intra and inter-rater agreement and correlation were estimated. RESULTS The results showed that agreements and correlations between experienced examiners were high with low variability. In addition, group analysis was noted as possessing the potential to solve individual inconsistency bias. CONCLUSION Furthermore, this study demonstrated the need for a group with prior training and experience, along with indicating the importance for the development of a clinical protocol that can use telemedicine for the evaluation of individuals with PD, as well as the inclusion of a specialized mediating group. In Addition, this research helps to the development of a valid remote assessment of bradykinesia.
Collapse
Affiliation(s)
- L M D Luiz
- Centre for Innovation and Technology Assessment in Health, Faculty of Electrical Engineering, Federal University of Uberlândia, Uberlândia, Brazil.
| | - I A Marques
- Centre for Innovation and Technology Assessment in Health, Faculty of Electrical Engineering, Federal University of Uberlândia, Uberlândia, Brazil
| | - J P Folador
- Centre for Innovation and Technology Assessment in Health, Faculty of Electrical Engineering, Federal University of Uberlândia, Uberlândia, Brazil
| | - A O Andrade
- Centre for Innovation and Technology Assessment in Health, Faculty of Electrical Engineering, Federal University of Uberlândia, Uberlândia, Brazil
| |
Collapse
|
43
|
Lecomte CG, Audet J, Harnie J, Frigon A. A Validation of Supervised Deep Learning for Gait Analysis in the Cat. Front Neuroinform 2021; 15:712623. [PMID: 34489668 PMCID: PMC8417424 DOI: 10.3389/fninf.2021.712623] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/20/2021] [Accepted: 07/27/2021] [Indexed: 11/13/2022] Open
Abstract
Gait analysis in cats and other animals is generally performed with custom-made or commercially developed software to track reflective markers placed on bony landmarks. This often involves costly motion tracking systems. However, deep learning, and in particular DeepLabCutTM (DLC), allows motion tracking without requiring placing reflective markers or an expensive system. The purpose of this study was to validate the accuracy of DLC for gait analysis in the adult cat by comparing results obtained with DLC and a custom-made software (Expresso) that has been used in several cat studies. Four intact adult cats performed tied-belt (both belts at same speed) and split-belt (belts operating at different speeds) locomotion at different speeds and left-right speed differences on a split-belt treadmill. We calculated several kinematic variables, such as step/stride lengths and joint angles from the estimates made by the two software and assessed the agreement between the two measurements using intraclass correlation coefficient or Lin's concordance correlation coefficient as well as Pearson's correlation coefficients. The results showed that DLC is at least as precise as Expresso with good to excellent agreement for all variables. Indeed, all 12 variables showed an agreement above 0.75, considered good, while nine showed an agreement above 0.9, considered excellent. Therefore, deep learning, specifically DLC, is valid for measuring kinematic variables during locomotion in cats, without requiring reflective markers and using a relatively low-cost system.
Collapse
Affiliation(s)
- Charly G Lecomte
- Department of Pharmacology-Physiology, Faculty of Medicine and Health Sciences, Centre de Recherche du CHUS, Université de Sherbrooke, Sherbrooke, QC, Canada
| | - Johannie Audet
- Department of Pharmacology-Physiology, Faculty of Medicine and Health Sciences, Centre de Recherche du CHUS, Université de Sherbrooke, Sherbrooke, QC, Canada
| | - Jonathan Harnie
- Department of Pharmacology-Physiology, Faculty of Medicine and Health Sciences, Centre de Recherche du CHUS, Université de Sherbrooke, Sherbrooke, QC, Canada
| | - Alain Frigon
- Department of Pharmacology-Physiology, Faculty of Medicine and Health Sciences, Centre de Recherche du CHUS, Université de Sherbrooke, Sherbrooke, QC, Canada
| |
Collapse
|
44
|
Simonet C, Galmes MA, Lambert C, Rees RN, Haque T, Bestwick JP, Lees AJ, Schrag A, Noyce AJ. Slow Motion Analysis of Repetitive Tapping (SMART) Test: Measuring Bradykinesia in Recently Diagnosed Parkinson's Disease and Idiopathic Anosmia. JOURNAL OF PARKINSONS DISEASE 2021; 11:1901-1915. [PMID: 34180422 DOI: 10.3233/jpd-212683] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Abstract
BACKGROUND Bradykinesia is the defining motor feature of Parkinson's disease (PD). There are limitations to its assessment using standard clinical rating scales, especially in the early stages of PD when a floor effect may be observed. OBJECTIVE To develop a quantitative method to track repetitive tapping movements and to compare people in the early stages of PD, healthy controls, and individuals with idiopathic anosmia. METHODS This was a cross-sectional study of 99 participants (early-stage PD = 26, controls = 64, idiopathic anosmia = 9). For each participant, repetitive finger tapping was recorded over 20 seconds using a smartphone at 240 frames per second. From each video, amplitude between fingers, frequency (number of taps per second), and velocity (distance travelled per second) was extracted. Clinical assessment was based on the motor section of the MDS-UPDRS. RESULTS People in the early stage of PD performed the task with slower velocity (p < 0.001) and with greater frequency slope than controls (p = 0.003). The combination of reduced velocity and greater frequency slope obtained the best accuracy to separate early-stage PD from controls based on metric thresholds alone (AUC = 0.88). Individuals with anosmia exhibited slower velocity (p = 0.001) and smaller amplitude (p < 0.001) compared with controls. CONCLUSION We present a simple, proof-of-concept method to detect early motor dysfunction in PD. Mean tap velocity appeared to be the best parameter to differentiate patients with PD from controls. Patients with anosmia also showed detectable differences in motor performance compared with controls which may suggest that some were in the prodromal phase of PD.
Collapse
Affiliation(s)
- Cristina Simonet
- Preventive Neurology Unit, Wolfson Institute of Preventive Medicine, Barts and The London School of Medicine and Dentistry, Queen Mary University of London, London, United Kingdom
| | - Miquel A Galmes
- Physical and Analytical Chemistry Department, Jaume I University, Castelló de la Plana, Spain
| | | | - Richard N Rees
- Department of Clinical and Movement Neuroscience, UCL Institute of Neurology, London, United Kingdom
| | - Tahrina Haque
- Preventive Neurology Unit, Wolfson Institute of Preventive Medicine, Barts and The London School of Medicine and Dentistry, Queen Mary University of London, London, United Kingdom
| | - Jonathan P Bestwick
- Preventive Neurology Unit, Wolfson Institute of Preventive Medicine, Barts and The London School of Medicine and Dentistry, Queen Mary University of London, London, United Kingdom
| | - Andrew J Lees
- Reta Lila Weston Institute of Neurological Studies, University College London Queen Square Institute of Neurology, London, United Kingdom
| | - Anette Schrag
- Preventive Neurology Unit, Wolfson Institute of Preventive Medicine, Barts and The London School of Medicine and Dentistry, Queen Mary University of London, London, United Kingdom.,Department of Clinical and Movement Neuroscience, UCL Institute of Neurology, London, United Kingdom
| | - Alastair J Noyce
- Preventive Neurology Unit, Wolfson Institute of Preventive Medicine, Barts and The London School of Medicine and Dentistry, Queen Mary University of London, London, United Kingdom.,Department of Clinical and Movement Neuroscience, UCL Institute of Neurology, London, United Kingdom
| |
Collapse
|
45
|
Hausmann SB, Vargas AM, Mathis A, Mathis MW. Measuring and modeling the motor system with machine learning. Curr Opin Neurobiol 2021; 70:11-23. [PMID: 34116423 DOI: 10.1016/j.conb.2021.04.004] [Citation(s) in RCA: 17] [Impact Index Per Article: 5.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/01/2021] [Revised: 03/23/2021] [Accepted: 04/18/2021] [Indexed: 12/11/2022]
Abstract
The utility of machine learning in understanding the motor system is promising a revolution in how to collect, measure, and analyze data. The field of movement science already elegantly incorporates theory and engineering principles to guide experimental work, and in this review we discuss the growing use of machine learning: from pose estimation, kinematic analyses, dimensionality reduction, and closed-loop feedback, to its use in understanding neural correlates and untangling sensorimotor systems. We also give our perspective on new avenues, where markerless motion capture combined with biomechanical modeling and neural networks could be a new platform for hypothesis-driven research.
Collapse
Affiliation(s)
| | | | - Alexander Mathis
- EPFL, Swiss Federal Institute of Technology, Lausanne, Switzerland.
| | | |
Collapse
|
46
|
Sibley KG, Girges C, Hoque E, Foltynie T. Video-Based Analyses of Parkinson's Disease Severity: A Brief Review. JOURNAL OF PARKINSON'S DISEASE 2021; 11:S83-S93. [PMID: 33682727 PMCID: PMC8385513 DOI: 10.3233/jpd-202402] [Citation(s) in RCA: 23] [Impact Index Per Article: 7.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Accepted: 02/10/2021] [Indexed: 12/25/2022]
Abstract
Remote and objective assessment of the motor symptoms of Parkinson's disease is an area of great interest particularly since the COVID-19 crisis emerged. In this paper, we focus on a) the challenges of assessing motor severity via videos and b) the use of emerging video-based Artificial Intelligence (AI)/Machine Learning techniques to quantitate human movement and its potential utility in assessing motor severity in patients with Parkinson's disease. While we conclude that video-based assessment may be an accessible and useful way of monitoring motor severity of Parkinson's disease, the potential of video-based AI to diagnose and quantify disease severity in the clinical context is dependent on research with large, diverse samples, and further validation using carefully considered performance standards.
Collapse
Affiliation(s)
- Krista G. Sibley
- Department of Clinical and Movement Neurosciences, Institute of Neurology, University College London, London, UK
| | - Christine Girges
- Department of Clinical and Movement Neurosciences, Institute of Neurology, University College London, London, UK
| | - Ehsan Hoque
- Department of Computer Science, University of Rochester, Rochester, NY, USA
| | - Thomas Foltynie
- Department of Clinical and Movement Neurosciences, Institute of Neurology, University College London, London, UK
| |
Collapse
|
47
|
Zhao Z, Fang H, Williams S, Relton SD, Alty J, Casson AJ, Wong DC. Time series clustering to examine presence of decrement in Parkinson's finger-tapping bradykinesia. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2020; 2020:780-783. [PMID: 33018102 DOI: 10.1109/embc44109.2020.9175638] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
Parkinson's disease is diagnosed based on expert clinical observation of movements. One important clinical feature is decrement, whereby the range of finger motion decreases over the course of the observation. This decrement has been assumed to be linear but has not been examined closely.We previously developed a method to extract a time series representation of a finger-tapping clinical test from 137 smart- phone video recordings. Here, we show how the signal can be processed to visualize archetypal progression of decrement. We use k-means with features derived from dynamic time warping to compare similarity of time series. To generate the archetypal time series corresponding to each cluster, we apply both a simple arithmetic mean, and dynamic time warping barycenter averaging to the time series belonging to each cluster.Visual inspection of the cluster-average time series showed two main trends. These corresponded well with participants with no bradykinesia and participants with severe bradykinesia. The visualizations support the concept that decrement tends to present as a linear decrease in range of motion over time.Clinical relevance- Our work visually presents the archetypal types of bradykinesia amplitude decrement, as seen in the Parkinson's finger-tapping test. We found two main patterns, one corresponding to no bradykinesia, and the other showing linear decrement over time.
Collapse
|
48
|
Williams S, Fang H, Relton SD, Graham CD, Alty JE. Seeing the unseen: Could Eulerian video magnification aid clinician detection of subclinical Parkinson's tremor? J Clin Neurosci 2020; 81:101-104. [PMID: 33222895 DOI: 10.1016/j.jocn.2020.09.046] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/13/2020] [Revised: 07/26/2020] [Accepted: 09/15/2020] [Indexed: 12/13/2022]
Abstract
INTRODUCTION Eulerian magnification amplifies very small movements in video, revealing otherwise invisible motion. This raises the possibility that it could enable clinician visualisation of subclinical tremor using a standard camera. We tested whether Eulerian magnification of apparently atremulous hands reveals a Parkinsonian tremor more frequently in Parkinson's than in controls. METHOD We applied Eulerian magnification to smartphone video of 48 hands that appeared atremulous during recording (22 hands from 11 control participants, 26 hands from 17 idiopathic Parkinson's participants). Videos were rated for Parkinsonian tremor appearance (yes/no) before and after Eulerian magnification by three movement disorder specialist neurologists. RESULTS The proportion of hands correctly classified as Parkinsonian or not by clinicians was significantly higher after Eulerian magnification (OR = 2.67; CI = [1.39, 5.17]; p < 0.003). Parkinsonian-appearance tremors were seen after magnification in a number of control hands, but the proportion was greater in the Parkinson's hands. CONCLUSION Eulerian magnification slightly improves clinician ability to identify apparently atremulous hands as Parkinsonian. This suggests that some of the apparent tremor revealed may be subclinical Parkinson's (pathological) tremor, and Eulerian magnification may represent a first step towards contactless visualisation of such tremor. However, the technique also reveals apparent tremor in control hands. Therefore, our method needs additional elaboration and would not be of direct clinical use in its current iteration.
Collapse
Affiliation(s)
- Stefan Williams
- University of Leeds, Leeds Institute of Health Science, Leeds, UK; Leeds Teaching Hospitals NHS Trust, UK.
| | - Hui Fang
- Loughborough University, Department of Computer Science, Loughborough, UK
| | - Samuel D Relton
- University of Leeds, Leeds Institute of Health Science, Leeds, UK
| | | | - Jane E Alty
- University of Tasmania, Hobart, Australia; Leeds Teaching Hospitals NHS Trust, UK
| |
Collapse
|