1
|
Udayagiri R, Yin J, Cai X, Townsend W, Trivedi V, Shende R, Sowande OF, Prosser LA, Pikul JH, Johnson MJ. Towards an AI-driven soft toy for automatically detecting and classifying infant-toy interactions using optical force sensors. Front Robot AI 2024; 11:1325296. [PMID: 38533525 PMCID: PMC10963494 DOI: 10.3389/frobt.2024.1325296] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/21/2023] [Accepted: 01/29/2024] [Indexed: 03/28/2024] Open
Abstract
Introduction: It is crucial to identify neurodevelopmental disorders in infants early on for timely intervention to improve their long-term outcomes. Combining natural play with quantitative measurements of developmental milestones can be an effective way to swiftly and efficiently detect infants who are at risk of neurodevelopmental delays. Clinical studies have established differences in toy interaction behaviors between full-term infants and pre-term infants who are at risk for cerebral palsy and other developmental disorders. Methods: The proposed toy aims to improve the quantitative assessment of infant-toy interactions and fully automate the process of detecting those infants at risk of developing motor delays. This paper describes the design and development of a toy that uniquely utilizes a collection of soft lossy force sensors which are developed using optical fibers to gather play interaction data from infants laying supine in a gym. An example interaction database was created by having 15 adults complete a total of 2480 interactions with the toy consisting of 620 touches, 620 punches-"kick substitute," 620 weak grasps and 620 strong grasps. Results: The data is analyzed for patterns of interaction with the toy face using a machine learning model developed to classify the four interactions present in the database. Results indicate that the configuration of 6 soft force sensors on the face created unique activation patterns. Discussion: The machine learning algorithm was able to identify the distinct action types from the data, suggesting the potential usability of the toy. Next steps involve sensorizing the entire toy and testing with infants.
Collapse
Affiliation(s)
- Rithwik Udayagiri
- Rehabilitation Robotics Lab (A GRASP Lab), University of Pennsylvania, Philadelphia, PA, United States
- Pikul Research Group (A GRASP Lab), University of Pennsylvania, Philadelphia, PA, United States
| | - Jessica Yin
- Pikul Research Group (A GRASP Lab), University of Pennsylvania, Philadelphia, PA, United States
- Department of Mechanical Engineering and Applied Mechanics, University of Pennsylvania, Philadelphia, PA, United States
| | - Xinyao Cai
- Pikul Research Group (A GRASP Lab), University of Pennsylvania, Philadelphia, PA, United States
| | - William Townsend
- Pikul Research Group (A GRASP Lab), University of Pennsylvania, Philadelphia, PA, United States
| | - Varun Trivedi
- Rehabilitation Robotics Lab (A GRASP Lab), University of Pennsylvania, Philadelphia, PA, United States
- Department of Bioengineering, University of Pennsylvania, Philadelphia, PA, United States
| | - Rohan Shende
- Rehabilitation Robotics Lab (A GRASP Lab), University of Pennsylvania, Philadelphia, PA, United States
- Department of Mechanical Engineering and Applied Mechanics, University of Pennsylvania, Philadelphia, PA, United States
| | - O. Francis Sowande
- Department of Mechanical Engineering and Applied Mechanics, University of Pennsylvania, Philadelphia, PA, United States
- Department of Bioengineering, University of Pennsylvania, Philadelphia, PA, United States
| | - Laura A. Prosser
- Department of Pediatrics, University of Pennsylvania, Philadelphia, PA, United States
| | - James H. Pikul
- Pikul Research Group (A GRASP Lab), University of Pennsylvania, Philadelphia, PA, United States
- Department of Mechanical Engineering and Applied Mechanics, University of Pennsylvania, Philadelphia, PA, United States
- Department of Mechanical Engineering, University of Wisconsin-Madison, Madison, WI, United States
| | - Michelle J. Johnson
- Rehabilitation Robotics Lab (A GRASP Lab), University of Pennsylvania, Philadelphia, PA, United States
- Department of Bioengineering, University of Pennsylvania, Philadelphia, PA, United States
- Department of Physical Medicine and Rehabilitation, University of Pennsylvania, Philadelphia, PA, United States
| |
Collapse
|
2
|
Xie Q, Sheng B, Huang J, Zhang Q, Zhang Y. A Pilot Study of Compensatory Strategies for Reach-to-Grasp-Pen in Patients with Stroke. Appl Bionics Biomech 2022; 2022:6933043. [PMID: 36406892 PMCID: PMC9674425 DOI: 10.1155/2022/6933043] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/21/2022] [Revised: 09/15/2022] [Accepted: 09/27/2022] [Indexed: 09/08/2024] Open
Abstract
Coordinated reaching and grasping movements may be impaired in patients with poststroke hemiplegia. Patients frequently adopt compensatory strategies, which require investigation. This pilot study used kinematic parameters to examine compensatory strategies by assessing the reach-to-grasp-pen movements in patients with stroke and unaffected participants. Twelve patients with stroke with mild impairment (45.16 ± 12.62 years, 2.41 ± 1.97 months), twelve with moderate impairment (50.41 ± 12.92 years, 3.83 ± 3.58 months), and ten healthy individuals (20.6 ± 0.69 years) performed a reach-to-grasp-pen task. Kinematics parameters of upper limb and fingers, such as movement time, number of movement units, index of curvature, spectral arc length, trunk forward transition, trunk lateral transition, elbow extension, shoulder flexion, shoulder abduction, trunk rotation, arm-plane angle, the joint angles of interphalangeal joints of the thumb, index, middle, ring, and little fingers were examined in the study. These parameters were evaluated with two Microsoft Azure Kinect and Leap Motion, which belong to markerless motion capture systems. Patients with stroke showed longer reaching movement time, less smooth movement trajectories, and more trunk rotation (P < 0.05). In patients with stroke, the metacarpophalangeal joint (MCP) and proximal interphalangeal joint (PIP) of the thumb were flexed in the starting position; the MCP and PIP joints of the index finger in the stroke group were more extended during pen grasp; the range of motion of the MCP of the middle finger and the PIP joints of the middle, ring, and little fingers became greater, suggesting a larger peak aperture (P < 0.05). The more significant extension was observed in the index finger at the end of the grasp, suggesting inadequate flexion (P < 0.05). In clinical practice, the reach-to-grasp-pen task using markless sensing technology can effectively distinguish patients with stroke from healthy individuals and evaluate the recovery and compensation strategies of upper limb and hand functions. It can potentially become an evaluation tool in hospital and community scenes. Accurate identification of abnormal trunk, arm, and finger strategies is crucial for therapists to develop targeted upper limb treatment methods and evaluate treatment effects.
Collapse
Affiliation(s)
- Qiurong Xie
- College of Rehabilitation Medicine, Fujian University of Traditional Chinese Medicine, Fuzhou 350122, Fujian, China
- Key Laboratory of Orthopedics & Traumatology of Traditional Chinese Medicine and Rehabilitation (Fujian University of TCM), Ministry of Education, Fuzhou, China
| | - Bo Sheng
- School of Mechatronic Engineering and Automation, Shanghai University, Shanghai 200444, China
| | - Jia Huang
- College of Rehabilitation Medicine, Fujian University of Traditional Chinese Medicine, Fuzhou 350122, Fujian, China
- Key Laboratory of Orthopedics & Traumatology of Traditional Chinese Medicine and Rehabilitation (Fujian University of TCM), Ministry of Education, Fuzhou, China
| | - Qi Zhang
- College of Rehabilitation Medicine, Fujian University of Traditional Chinese Medicine, Fuzhou 350122, Fujian, China
- Key Laboratory of Orthopedics & Traumatology of Traditional Chinese Medicine and Rehabilitation (Fujian University of TCM), Ministry of Education, Fuzhou, China
| | - Yanxin Zhang
- Key Laboratory of Orthopedics & Traumatology of Traditional Chinese Medicine and Rehabilitation (Fujian University of TCM), Ministry of Education, Fuzhou, China
- Department of Exercise Sciences, The University of Auckland, Newmarket, Auckland 1142, New Zealand
| |
Collapse
|
3
|
Mennella C, Alloisio S, Novellino A, Viti F. Characteristics and Applications of Technology-Aided Hand Functional Assessment: A Systematic Review. SENSORS (BASEL, SWITZERLAND) 2021; 22:199. [PMID: 35009742 PMCID: PMC8749695 DOI: 10.3390/s22010199] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 11/17/2021] [Revised: 12/22/2021] [Accepted: 12/23/2021] [Indexed: 01/08/2023]
Abstract
Technology-aided hand functional assessment has received considerable attention in recent years. Its applications are required to obtain objective, reliable, and sensitive methods for clinical decision making. This systematic review aims to investigate and discuss characteristics of technology-aided hand functional assessment and their applications, in terms of the adopted sensing technology, evaluation methods and purposes. Based on the shortcomings of current applications, and opportunities offered by emerging systems, this review aims to support the design and the translation to clinical practice of technology-aided hand functional assessment. To this end, a systematic literature search was led, according to recommended PRISMA guidelines, in PubMed and IEEE Xplore databases. The search yielded 208 records, resulting into 23 articles included in the study. Glove-based systems, instrumented objects and body-networked sensor systems appeared from the search, together with vision-based motion capture systems, end-effector, and exoskeleton systems. Inertial measurement unit (IMU) and force sensing resistor (FSR) resulted the sensing technologies most used for kinematic and kinetic analysis. A lack of standardization in system metrics and assessment methods emerged. Future studies that pertinently discuss the pathophysiological content and clinimetrics properties of new systems are required for leading technologies to clinical acceptance.
Collapse
Affiliation(s)
- Ciro Mennella
- Institute of Biophysics, National Research Council, Via De Marini 6, 16149 Genova, Italy; (S.A.); (F.V.)
| | - Susanna Alloisio
- Institute of Biophysics, National Research Council, Via De Marini 6, 16149 Genova, Italy; (S.A.); (F.V.)
- ETT Spa, Via Sestri 37, 16154 Genova, Italy;
| | | | - Federica Viti
- Institute of Biophysics, National Research Council, Via De Marini 6, 16149 Genova, Italy; (S.A.); (F.V.)
| |
Collapse
|
4
|
Kuroiwa T, Nimura A, Takahashi Y, Sasaki T, Koyama T, Okawa A, Fujita K. Device Development for Detecting Thumb Opposition Impairment Using Carbon Nanotube-Based Strain Sensors. SENSORS 2020; 20:s20143998. [PMID: 32708416 PMCID: PMC7412202 DOI: 10.3390/s20143998] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 06/02/2020] [Revised: 07/07/2020] [Accepted: 07/15/2020] [Indexed: 12/03/2022]
Abstract
Research into hand-sensing is the focus of various fields, such as medical engineering and ergonomics. The thumb is essential in these studies, as there is great value in assessing its opposition function. However, evaluation methods in the medical field, such as physical examination and computed tomography, and existing sensing methods in the ergonomics field have various shortcomings. Therefore, we conducted a comparative study using a carbon nanotube-based strain sensor to assess whether opposition movement and opposition impairment can be detected in 20 hands of volunteers and 14 hands of patients with carpal tunnel syndrome while avoiding existing shortcomings. We assembled a measurement device with two sensors and attached it to the dorsal skin of the first carpometacarpal joint. We measured sensor expansion and calculated the correlation coefficient during thumb motion. The average correlation coefficient significantly increased in the patient group, and intrarater and interrater reliability were good. Thus, the device accurately detected thumb opposition impairment due to carpal tunnel syndrome, with superior sensitivity and specificity relative to conventional manual inspection, and may also detect opposition impairment due to various diseases. Additionally, in the future, it could be used as an easy, affordable, and accurate sensor in sensor gloves.
Collapse
Affiliation(s)
- Tomoyuki Kuroiwa
- Department of Orthopaedic and Spinal Surgery, Graduate School of Medical and Dental Sciences, Tokyo Medical and Dental University, Tokyo 113-8519, Japan; (T.K.); (T.S.); (T.K.); (A.O.)
| | - Akimoto Nimura
- Department of Functional Joint Anatomy, Graduate School of Medical and Dental Sciences, Tokyo Medical and Dental University, Tokyo 113-8519, Japan;
| | - Yu Takahashi
- AI Group, Department of 1st Research and Development, Yamaha Corporation, Shizuoka 430-0904, Japan;
| | - Toru Sasaki
- Department of Orthopaedic and Spinal Surgery, Graduate School of Medical and Dental Sciences, Tokyo Medical and Dental University, Tokyo 113-8519, Japan; (T.K.); (T.S.); (T.K.); (A.O.)
| | - Takafumi Koyama
- Department of Orthopaedic and Spinal Surgery, Graduate School of Medical and Dental Sciences, Tokyo Medical and Dental University, Tokyo 113-8519, Japan; (T.K.); (T.S.); (T.K.); (A.O.)
| | - Atsushi Okawa
- Department of Orthopaedic and Spinal Surgery, Graduate School of Medical and Dental Sciences, Tokyo Medical and Dental University, Tokyo 113-8519, Japan; (T.K.); (T.S.); (T.K.); (A.O.)
| | - Koji Fujita
- Department of Functional Joint Anatomy, Graduate School of Medical and Dental Sciences, Tokyo Medical and Dental University, Tokyo 113-8519, Japan;
- Correspondence: ; Tel.: +81-3-5803-5279
| |
Collapse
|