1
|
Kadambi A, Bandini A, Ramkalawan RD, Hitzig SL, Zariffa J. Designing an Egocentric Video-Based Dashboard to Report Hand Performance Measures for Outpatient Rehabilitation of Cervical Spinal Cord Injury. Top Spinal Cord Inj Rehabil 2023; 29:75-87. [PMID: 38174134 PMCID: PMC10759816 DOI: 10.46292/sci23-00015s] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/05/2024]
Abstract
Background Functional use of the upper extremities (UEs) is a top recovery priority for individuals with cervical spinal cord injury (cSCI), but the inability to monitor recovery at home and limitations in hand function outcome measures impede optimal recovery. Objectives We developed a framework using wearable cameras to monitor hand use at home and aimed to identify the best way to report information to clinicians. Methods A dashboard was iteratively developed with clinician (n = 7) input through focus groups and interviews, creating low-fidelity prototypes based on recurring feedback until no new information emerged. Affinity diagramming was used to identify themes and subthemes from interview data. User stories were developed and mapped to specific features to create a high-fidelity prototype. Results Useful elements identified for a dashboard reporting hand performance included summaries to interpret graphs, a breakdown of hand posture and activity to provide context, video snippets to qualitatively view hand use at home, patient notes to understand patient satisfaction or struggles, and time series graphing of metrics to measure trends over time. Conclusion Involving end-users in the design process and breaking down user requirements into user stories helped identify necessary interface elements for reporting hand performance metrics to clinicians. Clinicians recognized the dashboard's potential to monitor rehabilitation progress, provide feedback on hand use, and track progress over time. Concerns were raised about the implementation into clinical practice, therefore further inquiry is needed to determine the tool's feasibility and usefulness in clinical practice for individuals with UE impairments.
Collapse
Affiliation(s)
- Adesh Kadambi
- KITE – Toronto Rehabilitation Institute, University Health Network, Toronto, ON, Canada
- Institute of Biomedical Engineering, University of Toronto, Toronto, ON, Canada
| | - Andrea Bandini
- KITE – Toronto Rehabilitation Institute, University Health Network, Toronto, ON, Canada
- Health Science Interdisciplinary Center, Scuola Superiore Sant’Anna, Pisa, Italy
- The Biorobotics Institute, Scuola Superiore Sant’Anna, Pisa, Italy
- Department of Excellence in Robotics and AI, Scuola Superiore Sant’Anna, Pisa, Italy
| | - Ryan D. Ramkalawan
- KITE – Toronto Rehabilitation Institute, University Health Network, Toronto, ON, Canada
| | - Sander L. Hitzig
- Rehabilitation Sciences Institute, Temerty Faculty of Medicine, University of Toronto, Toronto, ON, Canada
- St. John’s Rehab Research Program, Sunnybrook Research Institute, Sunnybrook Health Sciences Centre, Toronto, ON, Canada
- Department of Occupational Science & Occupational Therapy, Temerty Faculty of Medicine, University of Toronto, Toronto, ON, Canada
| | - José Zariffa
- KITE – Toronto Rehabilitation Institute, University Health Network, Toronto, ON, Canada
- Institute of Biomedical Engineering, University of Toronto, Toronto, ON, Canada
- Rehabilitation Sciences Institute, Temerty Faculty of Medicine, University of Toronto, Toronto, ON, Canada
- Edward S. Rogers Sr. Department of Electrical and Computer Engineering, University of Toronto, Toronto, ON, Canada
| |
Collapse
|
2
|
Forbrigger S, DePaul VG, Davies TC, Morin E, Hashtrudi-Zaad K. Home-based upper limb stroke rehabilitation mechatronics: challenges and opportunities. Biomed Eng Online 2023; 22:67. [PMID: 37424017 DOI: 10.1186/s12938-023-01133-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/10/2023] [Accepted: 07/01/2023] [Indexed: 07/11/2023] Open
Abstract
Interest in home-based stroke rehabilitation mechatronics, which includes both robots and sensor mechanisms, has increased over the past 12 years. The COVID-19 pandemic has exacerbated the existing lack of access to rehabilitation for stroke survivors post-discharge. Home-based stroke rehabilitation devices could improve access to rehabilitation for stroke survivors, but the home environment presents unique challenges compared to clinics. The present study undertakes a scoping review of designs for at-home upper limb stroke rehabilitation mechatronic devices to identify important design principles and areas for improvement. Online databases were used to identify papers published 2010-2021 describing novel rehabilitation device designs, from which 59 publications were selected describing 38 unique designs. The devices were categorized and listed according to their target anatomy, possible therapy tasks, structure, and features. Twenty-two devices targeted proximal (shoulder and elbow) anatomy, 13 targeted distal (wrist and hand) anatomy, and three targeted the whole arm and hand. Devices with a greater number of actuators in the design were more expensive, with a small number of devices using a mix of actuated and unactuated degrees of freedom to target more complex anatomy while reducing the cost. Twenty-six of the device designs did not specify their target users' function or impairment, nor did they specify a target therapy activity, task, or exercise. Twenty-three of the devices were capable of reaching tasks, 6 of which included grasping capabilities. Compliant structures were the most common approach of including safety features in the design. Only three devices were designed to detect compensation, or undesirable posture, during therapy activities. Six of the 38 device designs mention consulting stakeholders during the design process, only two of which consulted patients specifically. Without stakeholder involvement, these designs risk being disconnected from user needs and rehabilitation best practices. Devices that combine actuated and unactuated degrees of freedom allow a greater variety and complexity of tasks while not significantly increasing their cost. Future home-based upper limb stroke rehabilitation mechatronic designs should provide information on patient posture during task execution, design with specific patient capabilities and needs in mind, and clearly link the features of the design to users' needs.
Collapse
Affiliation(s)
- Shane Forbrigger
- Department of Electrical and Computer Engineering, Queen's University, Kingston, Canada
| | - Vincent G DePaul
- School of Rehabilitation Therapy, Queen's University, Kingston, Canada
| | - T Claire Davies
- Department of Mechanical and Materials Engineering, Queen's University, Kingston, Canada
| | - Evelyn Morin
- Department of Electrical and Computer Engineering, Queen's University, Kingston, Canada
| | - Keyvan Hashtrudi-Zaad
- Department of Electrical and Computer Engineering, Queen's University, Kingston, Canada.
| |
Collapse
|
3
|
Visual Object Tracking in First Person Vision. Int J Comput Vis 2023; 131:259-283. [PMID: 36624862 PMCID: PMC9816211 DOI: 10.1007/s11263-022-01694-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/03/2021] [Accepted: 09/22/2022] [Indexed: 01/12/2023]
Abstract
The understanding of human-object interactions is fundamental in First Person Vision (FPV). Visual tracking algorithms which follow the objects manipulated by the camera wearer can provide useful information to effectively model such interactions. In the last years, the computer vision community has significantly improved the performance of tracking algorithms for a large variety of target objects and scenarios. Despite a few previous attempts to exploit trackers in the FPV domain, a methodical analysis of the performance of state-of-the-art trackers is still missing. This research gap raises the question of whether current solutions can be used "off-the-shelf" or more domain-specific investigations should be carried out. This paper aims to provide answers to such questions. We present the first systematic investigation of single object tracking in FPV. Our study extensively analyses the performance of 42 algorithms including generic object trackers and baseline FPV-specific trackers. The analysis is carried out by focusing on different aspects of the FPV setting, introducing new performance measures, and in relation to FPV-specific tasks. The study is made possible through the introduction of TREK-150, a novel benchmark dataset composed of 150 densely annotated video sequences. Our results show that object tracking in FPV poses new challenges to current visual trackers. We highlight the factors causing such behavior and point out possible research directions. Despite their difficulties, we prove that trackers bring benefits to FPV downstream tasks requiring short-term object tracking. We expect that generic object tracking will gain popularity in FPV as new and FPV-specific methodologies are investigated. Supplementary Information The online version contains supplementary material available at 10.1007/s11263-022-01694-6.
Collapse
|
4
|
Bandini A, Dousty M, Hitzig SL, Craven BC, Kalsi-Ryan S, Zariffa J. Measuring Hand Use in the Home after Cervical Spinal Cord Injury Using Egocentric Video. J Neurotrauma 2022; 39:1697-1707. [PMID: 35747948 DOI: 10.1089/neu.2022.0156] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/06/2023] Open
Abstract
Egocentric video has recently emerged as a potential solution for monitoring hand function in individuals living with tetraplegia in the community, especially for its ability to detect functional use in the home environment. The aim of this study was to develop and validate a wearable vision-based system for measuring hand use in the home among individuals living with tetraplegia. Several deep learning algorithms for detecting functional hand-object interactions were developed and compared. The most accurate algorithm was used to extract measures of hand function from 65 h of unscripted video recorded at home by 20 participants with tetraplegia. These measures were: the percentage of interaction time over total recording time (Perc); the average duration of individual interactions (Dur); and the number of interactions per hour (Num). To demonstrate the clinical validity of the technology, egocentric measures were correlated with validated clinical assessments of hand function and independence (Graded Redefined Assessment of Strength, Sensibility and Prehension [GRASSP], Upper Extremity Motor Score [UEMS], and Spinal Cord Independent Measure [SCIM]). Hand-object interactions were automatically detected with a median F1-score of 0.80 (0.67-0.87). Our results demonstrated that higher UEMS and better prehension were related to greater time spent interacting, whereas higher SCIM and better hand sensation resulted in a higher number of interactions performed during the egocentric video recordings. For the first time, measures of hand function automatically estimated in an unconstrained environment in individuals with tetraplegia have been validated against internationally accepted measures of hand function. Future work will necessitate a formal evaluation of the reliability and responsiveness of the egocentric-based performance measures for hand use.
Collapse
Affiliation(s)
- Andrea Bandini
- KITE Research Institute and Toronto, Ontario, Canada.,The BioRobotics Institute and Scuola Superiore Sant'Anna, Pisa, Italy.,Department of Excellence in Robotics and AI, Scuola Superiore Sant'Anna, Pisa, Italy
| | - Mehdy Dousty
- KITE Research Institute and Toronto, Ontario, Canada.,Institute of Biomedical Engineering, University of Toronto, Toronto, Ontario, Canada
| | - Sander L Hitzig
- Rehabilitation Sciences Institute, University of Toronto, Toronto, Ontario, Canada.,St. John's Rehab Research Program, Sunnybrook Research Institute, Sunnybrook Health Sciences Centre, Toronto, Ontario, Canada.,Department of Occupational Science and Occupational Therapy, and University of Toronto, Toronto, Ontario, Canada
| | - B Catharine Craven
- KITE Research Institute and Toronto, Ontario, Canada.,Brain and Spinal Cord Rehabilitation Program Toronto Rehabilitation Institute - University Health Network, Toronto, Ontario, Canada.,Division of Physical Medicine and Rehabilitation Temerty Faculty of Medicine, and University of Toronto, Toronto, Ontario, Canada
| | - Sukhvinder Kalsi-Ryan
- KITE Research Institute and Toronto, Ontario, Canada.,Department of Physical Therapy and University of Toronto, Toronto, Ontario, Canada
| | - José Zariffa
- KITE Research Institute and Toronto, Ontario, Canada.,Institute of Biomedical Engineering, University of Toronto, Toronto, Ontario, Canada.,Rehabilitation Sciences Institute, University of Toronto, Toronto, Ontario, Canada.,Edward S. Rogers Sr. Department of Electrical and Computer Engineering, University of Toronto, Toronto, Ontario, Canada
| |
Collapse
|
5
|
Mennella C, Alloisio S, Novellino A, Viti F. Characteristics and Applications of Technology-Aided Hand Functional Assessment: A Systematic Review. SENSORS (BASEL, SWITZERLAND) 2021; 22:199. [PMID: 35009742 PMCID: PMC8749695 DOI: 10.3390/s22010199] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 11/17/2021] [Revised: 12/22/2021] [Accepted: 12/23/2021] [Indexed: 01/08/2023]
Abstract
Technology-aided hand functional assessment has received considerable attention in recent years. Its applications are required to obtain objective, reliable, and sensitive methods for clinical decision making. This systematic review aims to investigate and discuss characteristics of technology-aided hand functional assessment and their applications, in terms of the adopted sensing technology, evaluation methods and purposes. Based on the shortcomings of current applications, and opportunities offered by emerging systems, this review aims to support the design and the translation to clinical practice of technology-aided hand functional assessment. To this end, a systematic literature search was led, according to recommended PRISMA guidelines, in PubMed and IEEE Xplore databases. The search yielded 208 records, resulting into 23 articles included in the study. Glove-based systems, instrumented objects and body-networked sensor systems appeared from the search, together with vision-based motion capture systems, end-effector, and exoskeleton systems. Inertial measurement unit (IMU) and force sensing resistor (FSR) resulted the sensing technologies most used for kinematic and kinetic analysis. A lack of standardization in system metrics and assessment methods emerged. Future studies that pertinently discuss the pathophysiological content and clinimetrics properties of new systems are required for leading technologies to clinical acceptance.
Collapse
Affiliation(s)
- Ciro Mennella
- Institute of Biophysics, National Research Council, Via De Marini 6, 16149 Genova, Italy; (S.A.); (F.V.)
| | - Susanna Alloisio
- Institute of Biophysics, National Research Council, Via De Marini 6, 16149 Genova, Italy; (S.A.); (F.V.)
- ETT Spa, Via Sestri 37, 16154 Genova, Italy;
| | | | - Federica Viti
- Institute of Biophysics, National Research Council, Via De Marini 6, 16149 Genova, Italy; (S.A.); (F.V.)
| |
Collapse
|
6
|
Abstract
OBJECTIVE Cervical spinal cord injury (cSCI) can impair motor function in the upper limbs. Video from wearable cameras (egocentric video) has the potential to provide monitoring of rehabilitation outcomes at home, but methods for automated analysis of this data are needed. Wrist flexion and extension are essential elements to track grasping strategies after cSCI, as they may reflect the use of the tenodesis grasp, a common compensatory strategy. However, there is no established method to evaluate wrist flexion and extension from egocentric video. METHODS We propose a machine-learning-based approach comprising three steps-hand detection, pose estimation, and arm orientation estimation-to estimate wrist angle data, leading to the detection of tenodesis grasp. RESULTS The hand detection in conjunction with the pose estimation algorithm correctly located wrist and index finger metacarpophalangeal coordinates in 63% and 76% of 15,319 annotated frames, respectively, extracted from egocentric videos of individuals with cSCI performing activities of daily living in a home simulation laboratory. The arm orientation algorithm had a mean absolute error of 2.76 +/- 0.39 degrees in 12,863 labeled frames. Using these estimates, the presence of a tenodesis grasp was correctly detected in 72% +/- 11% of frames in videos of 6 activities. CONCLUSION The results provided a clear indication of which participants relied on tenodesis grasp and which did not. SIGNIFICANCE This paradigm provides the first method that can enable clinicians and researchers to monitor the use of the tenodesis grasp by individuals with cSCI at home, with implications for remote therapeutic guidance.
Collapse
|
7
|
Bandini A, Kalsi-Ryan S, Craven BC, Zariffa J, Hitzig SL. Perspectives and recommendations of individuals with tetraplegia regarding wearable cameras for monitoring hand function at home: Insights from a community-based study. J Spinal Cord Med 2021; 44:S173-S184. [PMID: 33960874 PMCID: PMC8604485 DOI: 10.1080/10790268.2021.1920787] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/28/2022] Open
Abstract
CONTEXT Wearable cameras have great potential for producing novel outcome measures of upper limb (UL) function and guiding care in individuals with cervical spinal cord injury (cSCI) living in the community. However, little is known about the perspectives of individuals with cSCI on the potential adoption of this technology. OBJECTIVE To analyze feedback from individuals with cSCI regarding the use of wearable cameras to record daily activities at home, in order to define guidelines for improving the design of this technology and fostering its implementation to optimize UL rehabilitation. DESIGN Mixed-methods study. PARTICIPANTS Thirteen adults with cSCI C3-C8 AIS A-D impairment. MEASURES Interview including survey and semi-structured questions. RESULTS Participants felt that this technology can provide naturalistic information regarding hand use to clinicians and researchers, which in turn can lead to better assessments of UL function and optimized therapies. Participants described the technology as easy-to-use but often reported discomfort that prevented them from conducting long recordings of fully natural activities. Privacy concerns included the possibility to capture household members and personal information displayed on objects (e.g. smartphones). CONCLUSION We provide the first set of guidelines to help researchers and therapists understand which steps need to be taken to translate wearable cameras into outpatient care and community-based research for UL rehabilitation. These guidelines include miniaturized and easy-to-wear cameras, as well as multiple measures for preventing privacy concerns such as avoiding public spaces and providing control over the recordings (e.g. start and stop the recordings at any time, keep or delete a recording).
Collapse
Affiliation(s)
- Andrea Bandini
- KITE Research Institute, Toronto Rehabilitation Institute, University Health Network (UHN), Toronto, Ontario, Canada
| | - Sukhvinder Kalsi-Ryan
- KITE Research Institute, Toronto Rehabilitation Institute, University Health Network (UHN), Toronto, Ontario, Canada
- Department of Physical Therapy, University of Toronto, Toronto, Ontario, Canada
| | - B. Catharine Craven
- KITE Research Institute, Toronto Rehabilitation Institute, University Health Network (UHN), Toronto, Ontario, Canada
- Brain and Spinal Cord Rehabilitation Program, Toronto Rehabilitation Institute, University Health Network, Toronto, Ontario, Canada
- Division of Physical Medicine and Rehabilitation, Department of Medicine, Faculty of Medicine, University of Toronto, Toronto, Ontario, Canada
| | - José Zariffa
- KITE Research Institute, Toronto Rehabilitation Institute, University Health Network (UHN), Toronto, Ontario, Canada
- Institute of Biomedical Engineering, University of Toronto, Toronto, Ontario, Canada
- Edward S. Rogers Sr. Department of Electrical and Computer Engineering, University of Toronto, Toronto, Ontario, Canada
- Rehabilitation Sciences Institute, Faculty of Medicine, University of Toronto, Toronto, Ontario, Canada
| | - Sander L. Hitzig
- Rehabilitation Sciences Institute, Faculty of Medicine, University of Toronto, Toronto, Ontario, Canada
- St. John’s Rehab Research Program, Sunnybrook Research Institute, Sunnybrook Health Sciences Centre, Toronto, Ontario, Canada
- Department of Occupational Science & Occupational Therapy, University of Toronto, Toronto, Ontario, Canada
| |
Collapse
|
8
|
Tsai MF, Wang RH, Zariffa J. Identifying Hand Use and Hand Roles After Stroke Using Egocentric Video. IEEE JOURNAL OF TRANSLATIONAL ENGINEERING IN HEALTH AND MEDICINE-JTEHM 2021; 9:2100510. [PMID: 33889453 PMCID: PMC8055062 DOI: 10.1109/jtehm.2021.3072347] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 10/23/2020] [Revised: 02/16/2021] [Accepted: 04/01/2021] [Indexed: 12/26/2022]
Abstract
Objective: Upper limb (UL) impairment impacts quality of life, but is common after stroke. UL function evaluated in the clinic may not reflect use in activities of daily living (ADLs) after stroke, and current approaches for assessment at home rely on self-report and lack details about hand function. Wrist-worn accelerometers have been applied to capture UL use, but also fail to reveal details of hand function. In response, a wearable system is proposed consisting of egocentric cameras combined with computer vision approaches, in order to identify hand use (hand-object interactions) and the role of the more-affected hand (as stabilizer or manipulator) in unconstrained environments. Methods: Nine stroke survivors recorded their performance of ADLs in a home simulation laboratory using an egocentric camera. Motion, hand shape, colour, and hand size change features were generated and fed into random forest classifiers to detect hand use and classify hand roles. Leave-one-subject-out cross-validation (LOSOCV) and leave-one-task-out cross-validation (LOTOCV) were used to evaluate the robustness of the algorithms. Results: LOSOCV and LOTOCV F1-scores for more-affected hand use were 0.64 ± 0.24 and 0.76 ± 0.23, respectively. For less-affected hands, LOSOCV and LOTOCV F1-scores were 0.72 ± 0.20 and 0.82 ± 0.22. F1-scores for hand role classification were 0.70 ± 0.19 and 0.68 ± 0.23 in the more-affected hand for LOSOCV and LOTOCV, respectively, and 0.59 ± 0.23 and 0.65 ± 0.28 in the less-affected hand. Conclusion: The results demonstrate the feasibility of predicting hand use and the hand roles of stroke survivors from egocentric videos.
Collapse
Affiliation(s)
- Meng-Fen Tsai
- KITE, Toronto Rehabilitation Institute, University Health NetworkTorontoONM5G 2A2Canada.,Institute of Biomedical Engineering, University of TorontoTorontoONM5S 1A1Canada
| | - Rosalie H Wang
- KITE, Toronto Rehabilitation Institute, University Health NetworkTorontoONM5G 2A2Canada.,Department of Occupational Science and Occupational TherapyUniversity of TorontoTorontoONM5S 1A1Canada
| | - Jose Zariffa
- KITE, Toronto Rehabilitation Institute, University Health NetworkTorontoONM5G 2A2Canada.,Institute of Biomedical Engineering, University of TorontoTorontoONM5S 1A1Canada
| |
Collapse
|
9
|
Capturing hand use of individuals with spinal cord injury at home using egocentric video: a feasibility study. Spinal Cord Ser Cases 2021; 7:17. [PMID: 33674553 DOI: 10.1038/s41394-021-00382-w] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/17/2020] [Revised: 12/27/2020] [Accepted: 02/10/2021] [Indexed: 11/09/2022] Open
Abstract
STUDY DESIGN Feasibility study. OBJECTIVE The objective of this study is to explore the feasibility of capturing egocentric (first person) video recordings in the home of individuals with cervical spinal cord injury (SCI) for hand function evaluation. SETTING Community-based study in Toronto, Ontario, Canada. METHODS Three participants with SCI recorded activities of daily living (ADLs) at home without the presence of a researcher. Information regarding recording characteristics and compliance was obtained as well as structured and semi-structured interviews involving privacy, usefulness, and usability. A video processing algorithm capable of detecting interactions between the hand and objects was applied to the home recordings. RESULTS In all, 98.58 ± 1.05% of the obtained footage was usable and included four to eight unique activities over a span of 3-7 days. The interaction detection algorithm yielded an F1 score of 0.75 ± 0.15. CONCLUSIONS Capturing ADLs using an egocentric camera in the home environment after SCI is feasible. Considerations regarding privacy, ease of use of the devices, and scheduling of recordings are provided.
Collapse
|
10
|
Bandini A, Dousty M, Zariffa J. A wearable vision-based system for detecting hand-object interactions in individuals with cervical spinal cord injury: First results in the home environment. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2020; 2020:2159-2162. [PMID: 33018434 DOI: 10.1109/embc44109.2020.9176274] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
Cervical spinal cord injury (cSCI) causes the paralysis of upper and lower limbs and trunk, significantly reducing quality of life and community participation of the affected individuals. The functional use of the upper limbs is the top recovery priority of people with cSCI and wearable vision-based systems have recently been proposed to extract objective outcome measures that reflect hand function in a natural context. However, previous studies were conducted in a controlled environment and may not be indicative of the actual hand use of people with cSCI living in the community. Thus, we propose a deep learning algorithm for automatically detecting hand-object interactions in egocentric videos recorded by participants with cSCI during their daily activities at home. The proposed approach is able to detect hand-object interactions with good accuracy (F1-score up to 0.82), demonstrating the feasibility of this system in uncontrolled situations (e.g., unscripted activities and variable illumination). This result paves the way for the development of an automated tool for measuring hand function in people with cSCI living in the community.
Collapse
|