1
|
Mazzeo A, Uliano M, Mucci P, Penzotti M, Angelini L, Cini F, Craighero L, Controzzi M. Human manipulation strategy when changing object deformability and task properties. Sci Rep 2024; 14:15819. [PMID: 38982184 PMCID: PMC11233673 DOI: 10.1038/s41598-024-65551-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/25/2024] [Accepted: 06/20/2024] [Indexed: 07/11/2024] Open
Abstract
Robotic literature widely addresses deformable object manipulation, but few studies analyzed human manipulation accounting for different levels of deformability and task properties. We asked participants to grasp and insert rigid and deformable objects into holes with varying tolerances and depths, and we analyzed the grasping behavior, the reaching velocity profile, and completion times. Results indicated that the more deformable the object is, the nearer the grasping point is to the extremity to be inserted. For insertions in the long hole, the selection of the grasping point is a trade-off between task accuracy and the number of re-grasps required to complete the insertion. The compliance of the deformable object facilitates the alignment between the object and the hole. The reaching velocity profile when increasing deformability recalls the one observed when task accuracy and precision decrease. Identifying human strategy allows the implementation of human-inspired high-level reasoning algorithms for robotic manipulation.
Collapse
Affiliation(s)
- A Mazzeo
- The BioRobotics Institute, Scuola Superiore Sant'Anna, Pisa, Italy.
- Department of Excellence in Robotics and AI, Scuola Superiore Sant'Anna, Pisa, Italy.
| | - M Uliano
- The BioRobotics Institute, Scuola Superiore Sant'Anna, Pisa, Italy
- Department of Excellence in Robotics and AI, Scuola Superiore Sant'Anna, Pisa, Italy
| | - P Mucci
- The BioRobotics Institute, Scuola Superiore Sant'Anna, Pisa, Italy
- Department of Excellence in Robotics and AI, Scuola Superiore Sant'Anna, Pisa, Italy
| | - M Penzotti
- The BioRobotics Institute, Scuola Superiore Sant'Anna, Pisa, Italy
- Department of Excellence in Robotics and AI, Scuola Superiore Sant'Anna, Pisa, Italy
| | - L Angelini
- The BioRobotics Institute, Scuola Superiore Sant'Anna, Pisa, Italy
- Department of Excellence in Robotics and AI, Scuola Superiore Sant'Anna, Pisa, Italy
| | - F Cini
- The BioRobotics Institute, Scuola Superiore Sant'Anna, Pisa, Italy
- Department of Excellence in Robotics and AI, Scuola Superiore Sant'Anna, Pisa, Italy
| | - L Craighero
- Department of Neuroscience and Rehabilitation, University of Ferrara, Ferrara, Italy
| | - M Controzzi
- The BioRobotics Institute, Scuola Superiore Sant'Anna, Pisa, Italy.
- Department of Excellence in Robotics and AI, Scuola Superiore Sant'Anna, Pisa, Italy.
| |
Collapse
|
2
|
Sadeghi M, Sharif Razavian R, Bazzi S, Chowdhury RH, Batista AP, Loughlin PJ, Sternad D. Inferring control objectives in a virtual balancing task in humans and monkeys. eLife 2024; 12:RP88514. [PMID: 38738986 PMCID: PMC11090506 DOI: 10.7554/elife.88514] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/14/2024] Open
Abstract
Natural behaviors have redundancy, which implies that humans and animals can achieve their goals with different strategies. Given only observations of behavior, is it possible to infer the control objective that the subject is employing? This challenge is particularly acute in animal behavior because we cannot ask or instruct the subject to use a particular strategy. This study presents a three-pronged approach to infer an animal's control objective from behavior. First, both humans and monkeys performed a virtual balancing task for which different control strategies could be utilized. Under matched experimental conditions, corresponding behaviors were observed in humans and monkeys. Second, a generative model was developed that represented two main control objectives to achieve the task goal. Model simulations were used to identify aspects of behavior that could distinguish which control objective was being used. Third, these behavioral signatures allowed us to infer the control objective used by human subjects who had been instructed to use one control objective or the other. Based on this validation, we could then infer objectives from animal subjects. Being able to positively identify a subject's control objective from observed behavior can provide a powerful tool to neurophysiologists as they seek the neural mechanisms of sensorimotor coordination.
Collapse
Affiliation(s)
- Mohsen Sadeghi
- Department of Biology, Northeastern UniversityBostonUnited States
| | - Reza Sharif Razavian
- Department of Biology, Northeastern UniversityBostonUnited States
- Department of Electrical and Computer Engineering, Northeastern UniversityBostonUnited States
- Department of Mechanical Engineering, Northern Arizona UniversityFlagstaffUnited States
| | - Salah Bazzi
- Department of Biology, Northeastern UniversityBostonUnited States
- Department of Electrical and Computer Engineering, Northeastern UniversityBostonUnited States
- Institute for Experiential Robotics, Northeastern UniversityBostonUnited States
| | - Raeed H Chowdhury
- Department of Bioengineering, and Center for the Neural Basis of Cognition, University of PittsburghPittsburghUnited States
| | - Aaron P Batista
- Department of Bioengineering, and Center for the Neural Basis of Cognition, University of PittsburghPittsburghUnited States
| | - Patrick J Loughlin
- Department of Bioengineering, and Center for the Neural Basis of Cognition, University of PittsburghPittsburghUnited States
| | - Dagmar Sternad
- Department of Biology, Northeastern UniversityBostonUnited States
- Department of Electrical and Computer Engineering, Northeastern UniversityBostonUnited States
- Institute for Experiential Robotics, Northeastern UniversityBostonUnited States
- Department of Physics, Northeastern UniversityBostonUnited States
| |
Collapse
|
3
|
Sadeghi M, Razavian RS, Bazzi S, Chowdhury R, Batista A, Loughlin P, Sternad D. Inferring control objectives in a virtual balancing task in humans and monkeys. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.05.02.539055. [PMID: 37205497 PMCID: PMC10187212 DOI: 10.1101/2023.05.02.539055] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/21/2023]
Abstract
Natural behaviors have redundancy, which implies that humans and animals can achieve their goals with different control objectives. Given only observations of behavior, is it possible to infer the control strategy that the subject is employing? This challenge is particularly acute in animal behavior because we cannot ask or instruct the subject to use a particular control strategy. This study presents a threepronged approach to infer an animal's control strategy from behavior. First, both humans and monkeys performed a virtual balancing task for which different control objectives could be utilized. Under matched experimental conditions, corresponding behaviors were observed in humans and monkeys. Second, a generative model was developed that represented two main control strategies to achieve the task goal. Model simulations were used to identify aspects of behavior that could distinguish which control objective was being used. Third, these behavioral signatures allowed us to infer the control objective used by human subjects who had been instructed to use one control objective or the other. Based on this validation, we could then infer strategies from animal subjects. Being able to positively identify a subject's control objective from behavior can provide a powerful tool to neurophysiologists as they seek the neural mechanisms of sensorimotor coordination.
Collapse
Affiliation(s)
- Mohsen Sadeghi
- Department of Biology, Northeastern University
- Department of Electrical and Computer Engineering, Northeastern University
| | | | - Salah Bazzi
- Institute for Experiential Robotics, Northeastern University
| | - Raeed Chowdhury
- Department of Bioengineering, and Center for the Neural Basis of Cognition, University of Pittsburgh, PA, USA
| | - Aaron Batista
- Department of Bioengineering, and Center for the Neural Basis of Cognition, University of Pittsburgh, PA, USA
| | - Patrick Loughlin
- Department of Bioengineering, and Center for the Neural Basis of Cognition, University of Pittsburgh, PA, USA
| | - Dagmar Sternad
- Department of Biology, Northeastern University
- Department of Electrical and Computer Engineering, Northeastern University
- Institute for Experiential Robotics, Northeastern University
- Department of Physics, Northeastern University
| |
Collapse
|
4
|
Maselli A, Gordon J, Eluchans M, Lancia GL, Thiery T, Moretti R, Cisek P, Pezzulo G. Beyond simple laboratory studies: Developing sophisticated models to study rich behavior. Phys Life Rev 2023; 46:220-244. [PMID: 37499620 DOI: 10.1016/j.plrev.2023.07.006] [Citation(s) in RCA: 12] [Impact Index Per Article: 12.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/05/2023] [Accepted: 07/06/2023] [Indexed: 07/29/2023]
Abstract
Psychology and neuroscience are concerned with the study of behavior, of internal cognitive processes, and their neural foundations. However, most laboratory studies use constrained experimental settings that greatly limit the range of behaviors that can be expressed. While focusing on restricted settings ensures methodological control, it risks impoverishing the object of study: by restricting behavior, we might miss key aspects of cognitive and neural functions. In this article, we argue that psychology and neuroscience should increasingly adopt innovative experimental designs, measurement methods, analysis techniques and sophisticated computational models to probe rich, ecologically valid forms of behavior, including social behavior. We discuss the challenges of studying rich forms of behavior as well as the novel opportunities offered by state-of-the-art methodologies and new sensing technologies, and we highlight the importance of developing sophisticated formal models. We exemplify our arguments by reviewing some recent streams of research in psychology, neuroscience and other fields (e.g., sports analytics, ethology and robotics) that have addressed rich forms of behavior in a model-based manner. We hope that these "success cases" will encourage psychologists and neuroscientists to extend their toolbox of techniques with sophisticated behavioral models - and to use them to study rich forms of behavior as well as the cognitive and neural processes that they engage.
Collapse
Affiliation(s)
- Antonella Maselli
- Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy
| | - Jeremy Gordon
- University of California, Berkeley, Berkeley, CA, 94704, United States
| | - Mattia Eluchans
- Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy; University of Rome "La Sapienza", Rome, Italy
| | - Gian Luca Lancia
- Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy; University of Rome "La Sapienza", Rome, Italy
| | - Thomas Thiery
- Department of Psychology, University of Montréal, Montréal, Québec, Canada
| | - Riccardo Moretti
- Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy; University of Rome "La Sapienza", Rome, Italy
| | - Paul Cisek
- Department of Neuroscience, University of Montréal, Montréal, Québec, Canada
| | - Giovanni Pezzulo
- Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy.
| |
Collapse
|
5
|
Park SW, Cardinaux A, Crozier D, Russo M, Kjelgaard M, Sinha P, Sternad D. Developmental change in predictive motor abilities. iScience 2023; 26:106038. [PMID: 36824276 PMCID: PMC9941204 DOI: 10.1016/j.isci.2023.106038] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/02/2022] [Revised: 12/14/2022] [Accepted: 01/17/2023] [Indexed: 01/26/2023] Open
Abstract
Prediction is critical for successful interactions with a dynamic environment. To test the development of predictive processes over the life span, we designed a suite of interceptive tasks implemented as interactive video games. Four tasks involving interactions with a flying ball with titrated challenge quantified spatiotemporal aspects of prediction. For comparison, reaction time was assessed in a matching task. The experiments were conducted in a museum, where over 400 visitors across all ages participated, and in a laboratory with a focused age group. Results consistently showed that predictive ability improved with age to reach adult level by age 12. In contrast, reaction time continued to decrease into late adolescence. Inter-task correlations revealed that the tasks tested different aspects of predictive processes. This developmental progression complements recent findings on cerebellar and cortical maturation. Additionally, these results can serve as normative data to study predictive processes in individuals with neurodevelopmental conditions.
Collapse
Affiliation(s)
- Se-Woong Park
- Department of Kinesiology, University of Texas at San Antonio, San Antonio, TX 78249, USA
- Department of Biology, Northeastern University, Boston, MA 02115, USA
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA 02139, USA
| | - Annie Cardinaux
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA 02139, USA
| | - Dena Crozier
- Department of Biology, Northeastern University, Boston, MA 02115, USA
- Department of Physics, Northeastern University, Boston, MA 02115, USA
- Cleveland Clinic Lerner College of Medicine, Case Western Reserve University, Cleveland, OH 44195, USA
| | - Marta Russo
- Department of Neurology, Tor Vergata Polyclinc, Rome, Italy
- Laboratory of Neuromotor Physiology, IRCCS Santa Lucia Foundation, Rome, Italy
| | - Margaret Kjelgaard
- Department of Communication Sciences and Disorders, Bridgewater State University, Bridgewater, MA 02325, USA
| | - Pawan Sinha
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA 02139, USA
| | - Dagmar Sternad
- Department of Biology, Northeastern University, Boston, MA 02115, USA
- Department of Physics, Northeastern University, Boston, MA 02115, USA
- Department of Electrical & Computer Engineering, Northeastern University, Boston, MA 02115, USA
| |
Collapse
|
6
|
Catenacci Volpi N, Greaves M, Trendafilov D, Salge C, Pezzulo G, Polani D. Skilled motor control of an inverted pendulum implies low entropy of states but high entropy of actions. PLoS Comput Biol 2023; 19:e1010810. [PMID: 36608159 PMCID: PMC9851554 DOI: 10.1371/journal.pcbi.1010810] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/22/2022] [Revised: 01/19/2023] [Accepted: 12/12/2022] [Indexed: 01/07/2023] Open
Abstract
The mastery of skills, such as balancing an inverted pendulum, implies a very accurate control of movements to achieve the task goals. Traditional accounts of skilled action control that focus on either routinization or perceptual control make opposite predictions about the ways we achieve mastery. The notion of routinization emphasizes the decrease of the variance of our actions, whereas the notion of perceptual control emphasizes the decrease of the variance of the states we visit, but not of the actions we execute. Here, we studied how participants managed control tasks of varying levels of difficulty, which consisted of controlling inverted pendulums of different lengths. We used information-theoretic measures to compare the predictions of alternative accounts that focus on routinization and perceptual control, respectively. Our results indicate that the successful performance of the control task strongly correlates with the decrease of state variability and the increase of action variability. As postulated by perceptual control theory, the mastery of skilled pendulum control consists in achieving stable control of goals by flexible means.
Collapse
Affiliation(s)
- Nicola Catenacci Volpi
- Department of Computer Science, University of Hertfordshire, Hatfield, England, United Kingdom
- * E-mail:
| | - Martin Greaves
- Department of Computer Science, University of Hertfordshire, Hatfield, England, United Kingdom
| | - Dari Trendafilov
- Institute for Pervasive Computing, Johannes Kepler University, Linz, Austria
| | - Christoph Salge
- Department of Computer Science, University of Hertfordshire, Hatfield, England, United Kingdom
| | - Giovanni Pezzulo
- Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy
| | - Daniel Polani
- Department of Computer Science, University of Hertfordshire, Hatfield, England, United Kingdom
| |
Collapse
|
7
|
Cregg JM, Mirdamadi JL, Fortunato C, Okorokova EV, Kuper C, Nayeem R, Byun AJ, Avraham C, Buonocore A, Winner TS, Mildren RL. Highlights from the 31st Annual Meeting of the Society for the Neural Control of Movement. J Neurophysiol 2023; 129:220-234. [PMID: 36541602 PMCID: PMC9844973 DOI: 10.1152/jn.00500.2022] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/13/2022] [Accepted: 12/16/2022] [Indexed: 12/24/2022] Open
Affiliation(s)
- Jared M Cregg
- Department of Neuroscience, Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark
| | - Jasmine L Mirdamadi
- Division of Physical Therapy, Department of Rehabilitation Medicine, Emory University School of Medicine, Atlanta, Georgia
| | - Cátia Fortunato
- Department of Bioengineering, Imperial College London, London, United Kingdom
| | | | - Clara Kuper
- Department of Psychology, Humboldt-Universität zu Berlin, Berlin, Germany
- School of Mind and Brain, Humboldt-Universität zu Berlin, Berlin, Germany
| | - Rashida Nayeem
- Department of Electrical Engineering, Northeastern University, Boston, Massachusetts
| | - Andrew J Byun
- John A. Paulson School of Engineering and Applied Sciences, Harvard University, Cambridge, Massachusetts
| | - Chen Avraham
- Department of Biomedical Engineering, Ben-Gurion University of the Negev, Beersheva, Israel
| | - Antimo Buonocore
- Werner Reichardt Centre for Integrative Neuroscience, Hertie Institute for Clinical Brain Research, University of Tübingen, Tübingen, Germany
- Department of Educational, Psychological and Communication Sciences, Suor Orsola Benincasa University, Naples, Italy
| | - Taniel S Winner
- Wallace H. Coulter Department of Biomedical Engineering, Emory University and Georgia Institute of Technology, Atlanta, Georgia
| | - Robyn L Mildren
- Department of Biomedical Engineering, Johns Hopkins University School of Medicine, Baltimore, Maryland
| |
Collapse
|
8
|
Nayeem R, Sohn WJ, DiCarlo JA, Gochyyev P, Lin DJ, Sternad D. Novel Platform for Quantitative Assessment of Functional Object Interactions After Stroke. IEEE Trans Neural Syst Rehabil Eng 2022; 31:426-436. [PMID: 36455078 PMCID: PMC10079607 DOI: 10.1109/tnsre.2022.3226067] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/03/2022]
Abstract
Many persons with stroke exhibit upper extremity motor impairments. These impairments often lead to dysfunction and affect performance in activities of daily living, where successful manipulation of objects is essential. Hence, understanding how upper extremity motor deficits manifest in functional interactions with objects is critical for rehabilitation. However, quantifying skill in these tasks has been a challenge. Traditional rehabilitation assessments require highly trained clinicians, are time-consuming, and yield subjective scores. This paper introduces a custom-designed device, the "MAGIC Table", that can record real-time kinematics of persons with stroke during interaction with objects, specifically a 'cup of coffee'. The task and its quantitative assessments were derived from previous basic-science studies. Six participants after stroke and six able-bodied participants moved a 3D-printed cup with a rolling ball inside, representing sloshing coffee, with 3 levels of difficulty. Movements were captured via a high-resolution camera above the table. Conventional kinematic metrics (movement time and smoothness) and novel kinematic metrics accounting for object interaction (risk and predictability) evaluated performance. Expectedly, persons with stroke moved more slowly and less smoothly than able-bodied participants, in both simple reaches and during transport of the cup-and-ball system. However, the more sensitive metric was mutual information, which captured the predictability of interactions, essential in cup transport as shown in previous theoretical research. Predictability sensitively measured differences in performance with increasing levels of difficulty. It also showed the best intraclass consistency, promising sensitive differentiation between different levels of impairment. This first study highlights the feasibility of this new device and indicates that examining dynamic object interaction may provide valuable insights into upper extremity function after stroke useful for assessment and rehabilitation.
Collapse
Affiliation(s)
- Rashida Nayeem
- Department of Electrical and Computer Engineering, Northeastern University, Boston, MA, USA
| | - Won Joon Sohn
- Department of Electrical and Computer Engineering, Northeastern University, Boston, MA, USA
| | - Julie A. DiCarlo
- Department of Neurology, Massachusetts General Hospital, Center for Neurotechnology and Neurorecovery, Harvard Medical School, Boston, MA, USA
| | - Perman Gochyyev
- Department of Neurology, Massachusetts General Hospital, Center for Neurotechnology and Neurorecovery, Harvard Medical School, Boston, MA, USA
| | - David J. Lin
- Department of Neurology, Massachusetts General Hospital, Center for Neurotechnology and Neurorecovery, Harvard Medical School, Boston, MA, USA
| | - Dagmar Sternad
- Department of Electrical and Computer Engineering and the Department of Biology and Physics, Northeastern University, Boston, MA, USA
| |
Collapse
|
9
|
Krotov A, Russo M, Nah M, Hogan N, Sternad D. Motor control beyond reach-how humans hit a target with a whip. ROYAL SOCIETY OPEN SCIENCE 2022; 9:220581. [PMID: 36249337 PMCID: PMC9533004 DOI: 10.1098/rsos.220581] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/05/2022] [Accepted: 09/16/2022] [Indexed: 06/01/2023]
Abstract
Humans are strikingly adept at manipulating complex objects, from tying shoelaces to cracking a bullwhip. These motor skills have highly nonlinear interactive dynamics that defy reduction into parts. Yet, despite advances in data recording and processing, experiments in motor neuroscience still prioritize experimental reduction over realistic complexity. This study embraced the fully unconstrained behaviour of hitting a target with a 1.6-m bullwhip, both in rhythmic and discrete fashion. Adopting an object-centered approach to test the hypothesis that skilled movement simplifies the whip dynamics, the whip's evolution was characterized in relation to performance error and hand speed. Despite widely differing individual strategies, both discrete and rhythmic styles featured a cascade-like unfolding of the whip. Whip extension and orientation at peak hand speed predicted performance error, at least in the rhythmic style, suggesting that humans accomplished the task by setting initial conditions. These insights may inform further studies on human and robot control of complex objects.
Collapse
Affiliation(s)
- Aleksei Krotov
- Department of Bioengineering, Northeastern University, Boston, MA, USA
| | - Marta Russo
- Departments of Biology, Electrical and Computer Engineering, and Physics, Northeastern University, Boston, MA, USA
- Department of Neurology, Tor Vergata Polyclinic and Laboratory of Neuromotor Physiology, IRCCS Fondazione Santa Lucia, Rome, Italy
| | - Moses Nah
- Department of Mechanical Engineering, Massachusetts Institute of Technology, Cambridge, MA, USA
| | - Neville Hogan
- Department of Mechanical Engineering, Massachusetts Institute of Technology, Cambridge, MA, USA
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA, USA
| | - Dagmar Sternad
- Departments of Biology, Electrical and Computer Engineering, and Physics, Northeastern University, Boston, MA, USA
| |
Collapse
|