1
|
Kuzmina E, Kriukov D, Lebedev M. Neuronal travelling waves explain rotational dynamics in experimental datasets and modelling. Sci Rep 2024; 14:3566. [PMID: 38347042 PMCID: PMC10861525 DOI: 10.1038/s41598-024-53907-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/25/2023] [Accepted: 02/06/2024] [Indexed: 02/15/2024] Open
Abstract
Spatiotemporal properties of neuronal population activity in cortical motor areas have been subjects of experimental and theoretical investigations, generating numerous interpretations regarding mechanisms for preparing and executing limb movements. Two competing models, representational and dynamical, strive to explain the relationship between movement parameters and neuronal activity. A dynamical model uses the jPCA method that holistically characterizes oscillatory activity in neuron populations by maximizing the data rotational dynamics. Different rotational dynamics interpretations revealed by the jPCA approach have been proposed. Yet, the nature of such dynamics remains poorly understood. We comprehensively analyzed several neuronal-population datasets and found rotational dynamics consistently accounted for by a traveling wave pattern. For quantifying rotation strength, we developed a complex-valued measure, the gyration number. Additionally, we identified parameters influencing rotation extent in the data. Our findings suggest that rotational dynamics and traveling waves are typically the same phenomena, so reevaluation of the previous interpretations where they were considered separate entities is needed.
Collapse
Affiliation(s)
- Ekaterina Kuzmina
- Skolkovo Institute of Science and Technology, Vladimir Zelman Center for Neurobiology and Brain Rehabilitation, Moscow, Russia, 121205.
- Artificial Intelligence Research Institute (AIRI), Moscow, Russia.
| | - Dmitrii Kriukov
- Artificial Intelligence Research Institute (AIRI), Moscow, Russia
- Skolkovo Institute of Science and Technology, Center for Molecular and Cellular Biology, Moscow, Russia, 121205
| | - Mikhail Lebedev
- Faculty of Mechanics and Mathematics, Lomonosov Moscow State University, Moscow, Russia, 119992
- Sechenov Institute of Evolutionary Physiology and Biochemistry of the Russian Academy of Sciences, Saint-Petersburg, Russia, 194223
| |
Collapse
|
2
|
Laurence-Chasen JD, Ross CF, Arce-McShane FI, Hatsopoulos NG. Robust cortical encoding of 3D tongue shape during feeding in macaques. Nat Commun 2023; 14:2991. [PMID: 37225708 PMCID: PMC10209084 DOI: 10.1038/s41467-023-38586-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/25/2022] [Accepted: 05/08/2023] [Indexed: 05/26/2023] Open
Abstract
Dexterous tongue deformation underlies eating, drinking, and speaking. The orofacial sensorimotor cortex has been implicated in the control of coordinated tongue kinematics, but little is known about how the brain encodes-and ultimately drives-the tongue's 3D, soft-body deformation. Here we combine a biplanar x-ray video technology, multi-electrode cortical recordings, and machine-learning-based decoding to explore the cortical representation of lingual deformation. We trained long short-term memory (LSTM) neural networks to decode various aspects of intraoral tongue deformation from cortical activity during feeding in male Rhesus monkeys. We show that both lingual movements and complex lingual shapes across a range of feeding behaviors could be decoded with high accuracy, and that the distribution of deformation-related information across cortical regions was consistent with previous studies of the arm and hand.
Collapse
Affiliation(s)
- Jeffrey D Laurence-Chasen
- Department of Organismal Biology and Anatomy, The University of Chicago, 1027 E 57th Street, Chicago, IL, 60637, USA.
| | - Callum F Ross
- Department of Organismal Biology and Anatomy, The University of Chicago, 1027 E 57th Street, Chicago, IL, 60637, USA
| | - Fritzie I Arce-McShane
- Department of Oral Health Sciences, School of Dentistry, University of Washington, 1959 NE Pacific Street, Box #357475, Seattle, WA, 98195-7475, USA
- Graduate Program in Neuroscience, University of Washington, 1959 NE Pacific St., Seattle, WA, 98195-7475, USA
| | - Nicholas G Hatsopoulos
- Department of Organismal Biology and Anatomy, The University of Chicago, 1027 E 57th Street, Chicago, IL, 60637, USA
- Program in Computational Neuroscience, The University of Chicago, 5812 South Ellis Avenue, Chicago, IL, 60637, USA
| |
Collapse
|
3
|
Moreira J, Silva B, Faria H, Santos R, Sousa ASP. Systematic Review on the Applicability of Principal Component Analysis for the Study of Movement in the Older Adult Population. SENSORS (BASEL, SWITZERLAND) 2022; 23:205. [PMID: 36616803 PMCID: PMC9823400 DOI: 10.3390/s23010205] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 10/24/2022] [Revised: 11/28/2022] [Accepted: 12/22/2022] [Indexed: 06/17/2023]
Abstract
Principal component analysis (PCA) is a dimensionality reduction method that has identified significant differences in older adults' motion analysis previously not detected by the discrete exploration of biomechanical variables. This systematic review aims to synthesize the current evidence regarding PCA use in the study of movement in older adults (kinematics and kinetics), summarizing the tasks and biomechanical variables studied. From the search results, 1685 studies were retrieved, and 19 studies were included for review. Most of the included studies evaluated gait or quiet standing. The main variables considered included spatiotemporal parameters, range of motion, and ground reaction forces. A limited number of studies analyzed other tasks. Further research should focus on the PCA application in tasks other than gait to understand older adults' movement characteristics that have not been identified by discrete analysis.
Collapse
Affiliation(s)
- Juliana Moreira
- Center for Rehabilitation Research–Human Movement System (Re)habilitation Area, Department of Physiotherapy, School of Health, Polytechnic of Porto, 4200-072 Porto, Portugal
- Research Center in Physical Activity, Health and Leisure, Faculty of Sports, University of Porto, 4200-450 Porto, Portugal
| | - Bruno Silva
- School of Health, Polytechnic of Porto, 4200-072 Porto, Portugal
| | - Hugo Faria
- School of Health, Polytechnic of Porto, 4200-072 Porto, Portugal
| | - Rubim Santos
- Center for Rehabilitation Research–Human Movement System (Re)habilitation Area, Department of Physics, School of Health, Polytechnic of Porto, 4200-072 Porto, Portugal
| | - Andreia S. P. Sousa
- Center for Rehabilitation Research–Human Movement System (Re)habilitation Area, Department of Physiotherapy, School of Health, Polytechnic of Porto, 4200-072 Porto, Portugal
| |
Collapse
|
4
|
Ventral premotor cortex encodes task relevant features during eye and head movements. Sci Rep 2022; 12:22093. [PMID: 36543870 PMCID: PMC9772313 DOI: 10.1038/s41598-022-26479-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/11/2022] [Accepted: 12/15/2022] [Indexed: 12/24/2022] Open
Abstract
Visual exploration of the environment is achieved through gaze shifts or coordinated movements of the eyes and the head. The kinematics and contributions of each component can be decoupled to fit the context of the required behavior, such as redirecting the visual axis without moving the head or rotating the head without changing the line of sight. A neural controller of these effectors, therefore, must show code relating to multiple muscle groups, and it must also differentiate its code based on context. In this study we tested whether the ventral premotor cortex (PMv) in monkey exhibits a population code relating to various features of eye and head movements. We constructed three different behavioral tasks or contexts, each with four variables to explore whether PMv modulates its activity in accordance with these factors. We found that task related population code in PMv differentiates between all task related features and conclude that PMv carries information about task relevant features during eye and head movements. Furthermore, this code represents both lower-level (effector and movement direction) and higher-level (context) information.
Collapse
|
5
|
Pei D, Olikkal P, Adali T, Vinjamuri R. Reconstructing Synergy-Based Hand Grasp Kinematics from Electroencephalographic Signals. SENSORS (BASEL, SWITZERLAND) 2022; 22:5349. [PMID: 35891029 PMCID: PMC9318424 DOI: 10.3390/s22145349] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 06/13/2022] [Revised: 07/15/2022] [Accepted: 07/16/2022] [Indexed: 06/15/2023]
Abstract
Brain-machine interfaces (BMIs) have become increasingly popular in restoring the lost motor function in individuals with disabilities. Several research studies suggest that the CNS may employ synergies or movement primitives to reduce the complexity of control rather than controlling each DoF independently, and the synergies can be used as an optimal control mechanism by the CNS in simplifying and achieving complex movements. Our group has previously demonstrated neural decoding of synergy-based hand movements and used synergies effectively in driving hand exoskeletons. In this study, ten healthy right-handed participants were asked to perform six types of hand grasps representative of the activities of daily living while their neural activities were recorded using electroencephalography (EEG). From half of the participants, hand kinematic synergies were derived, and a neural decoder was developed, based on the correlation between hand synergies and corresponding cortical activity, using multivariate linear regression. Using the synergies and the neural decoder derived from the first half of the participants and only cortical activities from the remaining half of the participants, their hand kinematics were reconstructed with an average accuracy above 70%. Potential applications of synergy-based BMIs for controlling assistive devices in individuals with upper limb motor deficits, implications of the results in individuals with stroke and the limitations of the study were discussed.
Collapse
|
6
|
DePass M, Falaki A, Quessy S, Dancause N, Cos I. A machine learning approach to characterize sequential movement-related states in premotor and motor cortices. J Neurophysiol 2022; 127:1348-1362. [PMID: 35171745 DOI: 10.1152/jn.00368.2021] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/22/2022] Open
Abstract
Non-human primate (NHP) movement kinematics have been decoded from spikes and local field potentials (LFPs) recorded during motor tasks. However, the potential of LFPs to provide network-like characterizations of neural dynamics during planning and execution of sequential movements requires further exploration. Is the aggregate nature of LFPs suitable to construct informative brain state descriptors of movement preparation and execution? To investigate this, we developed a framework to process LFPs based on machine learning classifiers and analyzed LFP from a primate, implanted with several microelectrode arrays covering the premotor cortex in both hemispheres and the primary motor cortex on one side. The primate performed a reach and grasp task, consisting of five consecutive states, starting from rest until a rewarding target (food) was attained. We use this five-state task to characterize brain activity and connectivity within eight frequency bands, using spectral amplitude and pair-wise correlations across electrodes as features. Our results show that we could best distinguish all five movement-related states using the highest frequency band (200-500Hz), yielding an 87% accuracy with spectral amplitude, and 60% with pair-wise electrode correlation. Further analyses characterized each movement-related state, showing differential neuronal population activity at above-gamma frequencies during the various stages of movement. In summary, our results show that the concerted use of novel machine-learning techniques with coarse grained queue broad signals such as LFPs may be successfully used to track and decode fine reach and grasp movement aspects across several brain regions.
Collapse
Affiliation(s)
- Michael DePass
- Departament de Matemàtiques i Informàtica, Facultat de Matemàtiques i Informàtica, Universitat de Barcelona, Barcelona, Catalonia, Spain
| | - Ali Falaki
- Département de Neurosciences, Faculté de Médecine, Université de Montréal, Québec, Canada, Canada
| | - Stephan Quessy
- Département de Neurosciences, Faculté de Médecine, Université de Montréal, Québec, Canada, Canada
| | - Numa Dancause
- Département de Neurosciences, Faculté de Médecine, Université de Montréal, Québec, Canada, Canada.,Centre interdisciplinaire de recherche sur le cerveau et l'apprentissage (CIRCA), Université de Montréal, Montreal, Québec, Canada
| | - Ignasi Cos
- Departament de Matemàtiques i Informàtica, Facultat de Matemàtiques i Informàtica, Universitat de Barcelona, Barcelona, Catalonia, Spain.,Serra-Húnter Research Programme, Barcelona, Catalonia, Spain
| |
Collapse
|
7
|
Nason SR, Mender MJ, Vaskov AK, Willsey MS, Ganesh Kumar N, Kung TA, Patil PG, Chestek CA. Real-time linear prediction of simultaneous and independent movements of two finger groups using an intracortical brain-machine interface. Neuron 2021; 109:3164-3177.e8. [PMID: 34499856 DOI: 10.1016/j.neuron.2021.08.009] [Citation(s) in RCA: 18] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/27/2020] [Revised: 06/07/2021] [Accepted: 08/10/2021] [Indexed: 11/27/2022]
Abstract
Modern brain-machine interfaces can return function to people with paralysis, but current upper extremity brain-machine interfaces are unable to reproduce control of individuated finger movements. Here, for the first time, we present a real-time, high-speed, linear brain-machine interface in nonhuman primates that utilizes intracortical neural signals to bridge this gap. We created a non-prehensile task that systematically individuates two finger groups, the index finger and the middle-ring-small fingers combined. During online brain control, the ReFIT Kalman filter could predict individuated finger group movements with high performance. Next, training ridge regression decoders with individual movements was sufficient to predict untrained combined movements and vice versa. Finally, we compared the postural and movement tuning of finger-related cortical activity to find that individual cortical units simultaneously encode multiple behavioral dimensions. Our results suggest that linear decoders may be sufficient for brain-machine interfaces to execute high-dimensional tasks with the performance levels required for naturalistic neural prostheses.
Collapse
Affiliation(s)
- Samuel R Nason
- Department of Biomedical Engineering, University of Michigan, Ann Arbor, MI 48109, USA
| | - Matthew J Mender
- Department of Biomedical Engineering, University of Michigan, Ann Arbor, MI 48109, USA
| | - Alex K Vaskov
- Robotics Graduate Program, University of Michigan, Ann Arbor, MI 48109, USA
| | - Matthew S Willsey
- Department of Biomedical Engineering, University of Michigan, Ann Arbor, MI 48109, USA; Department of Neurosurgery, University of Michigan Medical School, Ann Arbor, MI 48109, USA
| | - Nishant Ganesh Kumar
- Section of Plastic Surgery, Department of Surgery, University of Michigan, Ann Arbor, MI 48109, USA
| | - Theodore A Kung
- Section of Plastic Surgery, Department of Surgery, University of Michigan, Ann Arbor, MI 48109, USA
| | - Parag G Patil
- Department of Biomedical Engineering, University of Michigan, Ann Arbor, MI 48109, USA; Department of Neurosurgery, University of Michigan Medical School, Ann Arbor, MI 48109, USA; Department of Neurology, University of Michigan Medical School, Ann Arbor, MI 48109, USA; Neuroscience Graduate Program, University of Michigan, Ann Arbor, MI 48109, USA
| | - Cynthia A Chestek
- Department of Biomedical Engineering, University of Michigan, Ann Arbor, MI 48109, USA; Robotics Graduate Program, University of Michigan, Ann Arbor, MI 48109, USA; Neuroscience Graduate Program, University of Michigan, Ann Arbor, MI 48109, USA; Department of Electrical Engineering and Computer Science, University of Michigan, Ann Arbor, MI 48109, USA.
| |
Collapse
|
8
|
Versteeg C, Chowdhury RH, Miller LE. Cuneate nucleus: The somatosensory gateway to the brain. CURRENT OPINION IN PHYSIOLOGY 2021; 20:206-215. [PMID: 33869911 DOI: 10.1016/j.cophys.2021.02.004] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/20/2022]
Abstract
Much remains unknown about the transformation of proprioceptive afferent input from the periphery to the cortex. Until recently, the only recordings from neurons in the cuneate nucleus (CN) were from anesthetized animals. We are beginning to learn more about how the sense of proprioception is transformed as it propagates centrally. Recent recordings from microelectrode arrays chronically implanted in CN have revealed that CN neurons with muscle-like properties have a greater sensitivity to active reaching movements than to passive limb displacement, and we find that these neurons have receptive fields that resemble single muscles. In this review, we focus on the varied uses of proprioceptive input and the possible role of CN in processing this information.
Collapse
Affiliation(s)
- Christopher Versteeg
- Department of Biomedical Engineering, McCormick School of Engineering, Northwestern 7 University, Evanston, IL, USA
| | - Raeed H Chowdhury
- Department of Bioengineering, Swanson School of Engineering, University of Pittsburgh, 10 Pittsburgh, PA, USA
| | - Lee E Miller
- Department of Biomedical Engineering, McCormick School of Engineering, Northwestern 7 University, Evanston, IL, USA.,Department of Physiology, Feinberg School of Medicine, Northwestern University, Chicago, 13 IL, USA.,Department of Physical Medicine and Rehabilitation, Feinberg School of Medicine, 16 Northwestern University, Chicago, IL, USA.,Shirley Ryan AbilityLab, Chicago, IL, USA
| |
Collapse
|
9
|
Hughes C, Herrera A, Gaunt R, Collinger J. Bidirectional brain-computer interfaces. BRAIN-COMPUTER INTERFACES 2020; 168:163-181. [DOI: 10.1016/b978-0-444-63934-9.00013-5] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/27/2022]
|
10
|
Goodman JM, Tabot GA, Lee AS, Suresh AK, Rajan AT, Hatsopoulos NG, Bensmaia S. Postural Representations of the Hand in the Primate Sensorimotor Cortex. Neuron 2019; 104:1000-1009.e7. [PMID: 31668844 PMCID: PMC7172114 DOI: 10.1016/j.neuron.2019.09.004] [Citation(s) in RCA: 25] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/07/2019] [Revised: 06/04/2019] [Accepted: 09/05/2019] [Indexed: 01/07/2023]
Abstract
Manual dexterity requires proprioceptive feedback about the state of the hand. To date, study of the neural basis of proprioception in the cortex has focused primarily on reaching movements to the exclusion of hand-specific behaviors such as grasping. To fill this gap, we record both time-varying hand kinematics and neural activity evoked in somatosensory and motor cortices as monkeys grasp a variety of objects. We find that neurons in the somatosensory cortex, as well as in the motor cortex, preferentially track time-varying postures of multi-joint combinations spanning the entire hand. This contrasts with neural responses during reaching movements, which preferentially track time-varying movement kinematics of the arm, such as velocity and speed of the limb, rather than its time-varying postural configuration. These results suggest different representations of arm and hand movements suited to the different functional roles of these two effectors.
Collapse
Affiliation(s)
- James M Goodman
- Committee on Computational Neuroscience, University of Chicago, Chicago, IL, USA
| | - Gregg A Tabot
- Committee on Computational Neuroscience, University of Chicago, Chicago, IL, USA
| | - Alex S Lee
- Committee on Computational Neuroscience, University of Chicago, Chicago, IL, USA
| | - Aneesha K Suresh
- Committee on Computational Neuroscience, University of Chicago, Chicago, IL, USA
| | - Alexander T Rajan
- Committee on Computational Neuroscience, University of Chicago, Chicago, IL, USA
| | - Nicholas G Hatsopoulos
- Committee on Computational Neuroscience, University of Chicago, Chicago, IL, USA; Department of Organismal Biology and Anatomy, University of Chicago, Chicago, IL, USA; Grossman Institute for Neuroscience, Quantitative Biology, and Human Behavior, University of Chicago, Chicago, IL, USA
| | - Sliman Bensmaia
- Committee on Computational Neuroscience, University of Chicago, Chicago, IL, USA; Department of Organismal Biology and Anatomy, University of Chicago, Chicago, IL, USA; Grossman Institute for Neuroscience, Quantitative Biology, and Human Behavior, University of Chicago, Chicago, IL, USA.
| |
Collapse
|
11
|
Vukovic N, Shtyrov Y. Learning with the wave of the hand: Kinematic and TMS evidence of primary motor cortex role in category-specific encoding of word meaning. Neuroimage 2019; 202:116179. [DOI: 10.1016/j.neuroimage.2019.116179] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/19/2019] [Revised: 09/05/2019] [Accepted: 09/06/2019] [Indexed: 12/14/2022] Open
|
12
|
Yuliza E, Amalia N, Rahmayanti HD, Munir MM, Khairurrijal K, Abdullah M. How human age affects the signature’s curvature, density and amplitude to wavelength ratio and its potential application for countering document falsification. AUST J FORENSIC SCI 2019. [DOI: 10.1080/00450618.2019.1664633] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/26/2022]
Affiliation(s)
- Elfi Yuliza
- Department of Physics, Bandung Institute of Technology, Bandung, Indonesia
| | - Nadya Amalia
- Department of Physics, Bandung Institute of Technology, Bandung, Indonesia
| | | | | | | | | |
Collapse
|
13
|
Sun G, Zhang S, Zhang Y, Xu K, Zhang Q, Zhao T, Zheng X. Effective Dimensionality Reduction for Visualizing Neural Dynamics by Laplacian Eigenmaps. Neural Comput 2019; 31:1356-1379. [PMID: 31113304 DOI: 10.1162/neco_a_01203] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/04/2022]
Abstract
With the development of neural recording technology, it has become possible to collect activities from hundreds or even thousands of neurons simultaneously. Visualization of neural population dynamics can help neuroscientists analyze large-scale neural activities efficiently. In this letter, Laplacian eigenmaps is applied to this task for the first time, and the experimental results show that the proposed method significantly outperforms the commonly used methods. This finding was confirmed by the systematic evaluation using nonhuman primate data, which contained the complex dynamics well suited for testing. According to our results, Laplacian eigenmaps is better than the other methods in various ways and can clearly visualize interesting biological phenomena related to neural dynamics.
Collapse
Affiliation(s)
- G Sun
- Qiushi Academy for Advanced Studies, Key Laboratory of Biomedical Engineering of Education Ministry, Zhejiang University, and Department of Biomedical Engineering, Zhejiang Provincial Key Laboratory of Cardio-Cerebral Vascular Detection Technology and Medicinal Effectiveness Appraisal, Zhejiang University, Hangzhou 310027, China
| | - S Zhang
- Qiushi Academy for Advanced Studies, Key Laboratory of Biomedical Engineering of Education Ministry, Zhejiang University, and Department of Biomedical Engineering, Zhejiang Provincial Key Laboratory of Cardio-Cerebral Vascular Detection Technology and Medicinal Effectiveness Appraisal, Zhejiang University, Hangzhou 310027, China
| | - Y Zhang
- Qiushi Academy for Advanced Studies, Key Laboratory of Biomedical Engineering of Education Ministry, Zhejiang University, and Department of Biomedical Engineering, Zhejiang Provincial Key Laboratory of Cardio-Cerebral Vascular Detection Technology and Medicinal Effectiveness Appraisal, Zhejiang University, Hangzhou 310027, China
| | - K Xu
- Qiushi Academy for Advanced Studies, Key Laboratory of Biomedical Engineering of Education Ministry, Zhejiang University, and Department of Biomedical Engineering, Zhejiang Provincial Key Laboratory of Cardio-Cerebral Vascular Detection Technology and Medicinal Effectiveness Appraisal, Zhejiang University, Hangzhou 310027, China
| | - Q Zhang
- Qiushi Academy for Advanced Studies, Key Laboratory of Biomedical Engineering of Education Ministry, Zhejiang University, and Department of Biomedical Engineering, Zhejiang Provincial Key Laboratory of Cardio-Cerebral Vascular Detection Technology and Medicinal Effectiveness Appraisal, Zhejiang University, Hangzhou 310027, China
| | - T Zhao
- Howard Hughes Medical Institute, Janelia Research Campus, Ashburn, VA 20147, U.S.A.
| | - X Zheng
- Qiushi Academy for Advanced Studies, Key Laboratory of Biomedical Engineering of Education Ministry, Zhejiang University, and Department of Biomedical Engineering, Zhejiang Provincial Key Laboratory of Cardio-Cerebral Vascular Detection Technology and Medicinal Effectiveness Appraisal, Zhejiang University, Hangzhou 310027, China
| |
Collapse
|
14
|
Castiello U, Dadda M. A review and consideration on the kinematics of reach-to-grasp movements in macaque monkeys. J Neurophysiol 2018; 121:188-204. [PMID: 30427765 DOI: 10.1152/jn.00598.2018] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/18/2022] Open
Abstract
The bases for understanding the neuronal mechanisms that underlie the control of reach-to-grasp movements among nonhuman primates, particularly macaques, has been widely studied. However, only a few kinematic descriptions of their prehensile actions are available. A thorough understanding of macaques' prehensile movements is manifestly critical, in light of their role in biomedical research as valuable models for studying neuromotor disorders and brain mechanisms, as well as for developing brain-machine interfaces to facilitate arm control. This article aims to review the current state of knowledge on the kinematics of grasping movements that macaques perform in naturalistic, seminaturalistic, and laboratory settings, to answer the following questions: Are kinematic signatures affected by the context within which the movement is performed? In what ways are kinematics of humans' and macaques' prehensile actions similar/dissimilar? Our analysis reflects the challenges involved in making comparisons across settings and species due to the heterogeneous picture in terms of the number of subjects, stimuli, conditions, and hands used. The kinematics of free-ranging macaques are characterized by distinctive features that are exhibited neither by macaques in laboratory setting nor by human subjects. The temporal incidence of key kinematic landmarks diverges significantly between species, indicating disparities in the overall organization of movement. Given such complexities, we attempt a synthesis of the extant body of evidence, intending to generate some significant implications for directions that future research might take to recognize the remaining gaps and pursue the insights and resolutions to generate an interpretation of movement kinematics that accounts for all settings and subjects.
Collapse
Affiliation(s)
- Umberto Castiello
- Department of General Psychology, University of Padova , Padua , Italy
| | - Marco Dadda
- Department of General Psychology, University of Padova , Padua , Italy
| |
Collapse
|
15
|
Vaskov AK, Irwin ZT, Nason SR, Vu PP, Nu CS, Bullard AJ, Hill M, North N, Patil PG, Chestek CA. Cortical Decoding of Individual Finger Group Motions Using ReFIT Kalman Filter. Front Neurosci 2018; 12:751. [PMID: 30455621 PMCID: PMC6231049 DOI: 10.3389/fnins.2018.00751] [Citation(s) in RCA: 24] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/21/2018] [Accepted: 09/28/2018] [Indexed: 01/01/2023] Open
Abstract
Objective: To date, many brain-machine interface (BMI) studies have developed decoding algorithms for neuroprostheses that provide users with precise control of upper arm reaches with some limited grasping capabilities. However, comparatively few have focused on quantifying the performance of precise finger control. Here we expand upon this work by investigating online control of individual finger groups. Approach: We have developed a novel training manipulandum for non-human primate (NHP) studies to isolate the movements of two specific finger groups: index and middle-ring-pinkie (MRP) fingers. We use this device in combination with the ReFIT (Recalibrated Feedback Intention-Trained) Kalman filter to decode the position of each finger group during a single degree of freedom task in two rhesus macaques with Utah arrays in motor cortex. The ReFIT Kalman filter uses a two-stage training approach that improves online control of upper arm tasks with substantial reductions in orbiting time, thus making it a logical first choice for precise finger control. Results: Both animals were able to reliably acquire fingertip targets with both index and MRP fingers, which they did in blocks of finger group specific trials. Decoding from motor signals online, the ReFIT Kalman filter reliably outperformed the standard Kalman filter, measured by bit rate, across all tested finger groups and movements by 31.0 and 35.2%. These decoders were robust when the manipulandum was removed during online control. While index finger movements and middle-ring-pinkie finger movements could be differentiated from each other with 81.7% accuracy across both subjects, the linear Kalman filter was not sufficient for decoding both finger groups together due to significant unwanted movement in the stationary finger, potentially due to co-contraction. Significance: To our knowledge, this is the first systematic and biomimetic separation of digits for continuous online decoding in a NHP as well as the first demonstration of the ReFIT Kalman filter improving the performance of precise finger decoding. These results suggest that novel nonlinear approaches, apparently not necessary for center out reaches or gross hand motions, may be necessary to achieve independent and precise control of individual fingers.
Collapse
Affiliation(s)
- Alex K Vaskov
- Robotics Graduate Program, University of Michigan, Ann Arbor, MI, United States
| | - Zachary T Irwin
- Department of Biomedical Engineering, University of Michigan, Ann Arbor, MI, United States.,Department of Neurology, University of Alabama, Birmingham, AL, United States
| | - Samuel R Nason
- Department of Biomedical Engineering, University of Michigan, Ann Arbor, MI, United States
| | - Philip P Vu
- Department of Biomedical Engineering, University of Michigan, Ann Arbor, MI, United States
| | - Chrono S Nu
- Department of Biomedical Engineering, University of Michigan, Ann Arbor, MI, United States
| | - Autumn J Bullard
- Department of Biomedical Engineering, University of Michigan, Ann Arbor, MI, United States
| | - Mackenna Hill
- Department of Biomedical Engineering, University of Michigan, Ann Arbor, MI, United States.,Department of Biomedical Engineering, Duke University, Durham, NC, United States
| | - Naia North
- Mechanical Engineering Department, University of Michigan, Ann Arbor, MI, United States
| | - Parag G Patil
- Department of Biomedical Engineering, University of Michigan, Ann Arbor, MI, United States.,Department of Neurosurgery, University of Michigan, Ann Arbor, MI, United States.,Department of Neurology, University of Michigan, Ann Arbor, MI, United States.,Neuroscience Graduate Program, University of Michigan, Ann Arbor, MI, United States
| | - Cynthia A Chestek
- Robotics Graduate Program, University of Michigan, Ann Arbor, MI, United States.,Department of Biomedical Engineering, University of Michigan, Ann Arbor, MI, United States.,Neuroscience Graduate Program, University of Michigan, Ann Arbor, MI, United States.,Department of Electrical Engineering and Computer Science, University of Michigan, Ann Arbor, MI, United States
| |
Collapse
|
16
|
Muscle Synergies Obtained from Comprehensive Mapping of the Cortical Forelimb Representation Using Stimulus Triggered Averaging of EMG Activity. J Neurosci 2018; 38:8759-8771. [PMID: 30150363 DOI: 10.1523/jneurosci.2519-17.2018] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/31/2017] [Revised: 07/16/2018] [Accepted: 08/20/2018] [Indexed: 01/01/2023] Open
Abstract
Neuromuscular control of voluntary movement may be simplified using muscle synergies similar to those found using non-negative matrix factorization. We recently identified synergies in electromyography (EMG) recordings associated with both voluntary movement and movement evoked by high-frequency long-duration intracortical microstimulation applied to the forelimb representation of the primary motor cortex (M1). The goal of this study was to use stimulus-triggered averaging (StTA) of EMG activity to investigate the synergy profiles and weighting coefficients associated with poststimulus facilitation, as synergies may be hard-wired into elemental cortical output modules and revealed by StTA. We applied StTA at low (LOW, ∼15 μA) and high intensities (HIGH, ∼110 μA) to 247 cortical locations of the M1 forelimb region in two male rhesus macaques while recording the EMG of 24 forelimb muscles. Our results show that 10-11 synergies accounted for 90% of the variation in poststimulus EMG facilitation peaks from the LOW-intensity StTA dataset while only 4-5 synergies were needed for the HIGH-intensity dataset. Synergies were similar across monkeys and current intensities. Most synergy profiles strongly activated only one or two muscles; all joints were represented and most, but not all, joint directions of motion were represented. Cortical maps of the synergy weighting coefficients suggest only a weak organization. StTA of M1 resulted in highly diverse muscle activations, suggestive of the limiting condition of requiring a synergy for each muscle to account for the patterns observed.SIGNIFICANCE STATEMENT Coordination of muscle activity and the neural origin of potential muscle synergies remains a fundamental question of neuroscience. We previously demonstrated that high-frequency long-duration intracortical microstimulation-evoked synergies were unrelated to voluntary movement synergies and were not clearly organized in the cortex. Here we present stimulus-triggered averaging facilitation-related muscle synergies, suggesting that when fundamental cortical output modules are activated, synergies approach the limit of single-muscle control. Thus, we conclude that if the CNS controls movement via linear synergies, those synergies are unlikely to be called from M1. This information is critical for understanding neural control of movement and the development of brain-machine interfaces.
Collapse
|
17
|
Smith RJ, Soares AB, Rouse AG, Schieber MH, Thakor NV. Modeling task-specific neuronal ensembles improves decoding of grasp. J Neural Eng 2018; 15:036006. [PMID: 29393065 DOI: 10.1088/1741-2552/aaac93] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/11/2022]
Abstract
OBJECTIVE Dexterous movement involves the activation and coordination of networks of neuronal populations across multiple cortical regions. Attempts to model firing of individual neurons commonly treat the firing rate as directly modulating with motor behavior. However, motor behavior may additionally be associated with modulations in the activity and functional connectivity of neurons in a broader ensemble. Accounting for variations in neural ensemble connectivity may provide additional information about the behavior being performed. APPROACH In this study, we examined neural ensemble activity in primary motor cortex (M1) and premotor cortex (PM) of two male rhesus monkeys during performance of a center-out reach, grasp and manipulate task. We constructed point process encoding models of neuronal firing that incorporated task-specific variations in the baseline firing rate as well as variations in functional connectivity with the neural ensemble. Models were evaluated both in terms of their encoding capabilities and their ability to properly classify the grasp being performed. MAIN RESULTS Task-specific ensemble models correctly predicted the performed grasp with over 95% accuracy and were shown to outperform models of neuronal activity that assume only a variable baseline firing rate. Task-specific ensemble models exhibited superior decoding performance in 82% of units in both monkeys (p < 0.01). Inclusion of ensemble activity also broadly improved the ability of models to describe observed spiking. Encoding performance of task-specific ensemble models, measured by spike timing predictability, improved upon baseline models in 62% of units. SIGNIFICANCE These results suggest that additional discriminative information about motor behavior found in the variations in functional connectivity of neuronal ensembles located in motor-related cortical regions is relevant to decode complex tasks such as grasping objects, and may serve the basis for more reliable and accurate neural prosthesis.
Collapse
Affiliation(s)
- Ryan J Smith
- Department of Biomedical Engineering, Johns Hopkins University, Baltimore, MD, United States of America
| | | | | | | | | |
Collapse
|
18
|
Hernández-Pérez R, Cuaya LV, Rojas-Hortelano E, Reyes-Aguilar A, Concha L, de Lafuente V. Tactile object categories can be decoded from the parietal and lateral-occipital cortices. Neuroscience 2017; 352:226-235. [DOI: 10.1016/j.neuroscience.2017.03.038] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/10/2016] [Revised: 03/23/2017] [Accepted: 03/24/2017] [Indexed: 01/08/2023]
|
19
|
Encoding of Both Reaching and Grasping Kinematics in Dorsal and Ventral Premotor Cortices. J Neurosci 2017; 37:1733-1746. [PMID: 28077725 DOI: 10.1523/jneurosci.1537-16.2016] [Citation(s) in RCA: 40] [Impact Index Per Article: 5.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/11/2016] [Revised: 12/12/2016] [Accepted: 12/30/2016] [Indexed: 01/05/2023] Open
Abstract
Classically, it has been hypothesized that reach-to-grasp movements arise from two discrete parietofrontal cortical networks. As part of these networks, the dorsal premotor cortex (PMd) has been implicated in the control of reaching movements of the arm, whereas the ventral premotor cortex (PMv) has been associated with the control of grasping movements of the hand. Recent studies have shown that such a strict delineation of function along anatomical boundaries is unlikely, partly because reaching to different locations can alter distal hand kinematics and grasping different objects can affect kinematics of the proximal arm. Here, we used chronically implanted multielectrode arrays to record unit-spiking activity in both PMd and PMv simultaneously while rhesus macaques engaged in a reach-to-grasp task. Generalized linear models were used to predict the spiking activity of cells in both areas as a function of different kinematic parameters, as well as spike history. To account for the influence of reaching on hand kinematics and vice versa, we applied demixed principal components analysis to define kinematics synergies that maximized variance across either different object locations or grip types. We found that single cells in both PMd and PMv encode the kinematics of both reaching and grasping synergies, suggesting that this classical division of reach and grasp in PMd and PMv, respectively, does not accurately reflect the encoding preferences of cells in those areas.SIGNIFICANCE STATEMENT For reach-to-grasp movements, the dorsal premotor cortex (PMd) has been implicated in the control of reaching movements of the arm, whereas the ventral premotor cortex (PMv) has been associated with the control of grasping movements of the hand. We recorded unit-spiking activity in PMd and PMv simultaneously while macaques performed a reach-to-grasp task. We modeled the spiking activity of neurons as a function of kinematic parameters and spike history. We applied demixed principal components analysis to define kinematics synergies. We found that single units in both PMd and PMv encode the kinematics of both reaching and grasping synergies, suggesting that the division of reach and grasp in PMd and PMv, respectively, cannot be made based on their encoding properties.
Collapse
|
20
|
Schwartz AB. Beyond synergies: Comment on "Hand synergies: Integration of robotics and neuroscience for understanding the control of biological and artificial hands" by Marco Santello et al. Phys Life Rev 2016; 17:50-3. [PMID: 27105943 DOI: 10.1016/j.plrev.2016.04.004] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/07/2016] [Accepted: 04/08/2016] [Indexed: 11/17/2022]
|
21
|
Schieber MH. Neuro-prosthetic interplay: Comment on "Hand synergies: Integration of robotics and neuroscience for understanding the control of biological and artificial hands" by M. Santello et al. Phys Life Rev 2016; 17:47-9. [PMID: 27105944 DOI: 10.1016/j.plrev.2016.03.004] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/22/2016] [Accepted: 03/29/2016] [Indexed: 10/22/2022]
Affiliation(s)
- Marc H Schieber
- Departments of Neurology, Neuroscience, and Biomedical Engineering, University of Rochester, Rochester, NY, USA.
| |
Collapse
|
22
|
Leo A, Handjaras G, Bianchi M, Marino H, Gabiccini M, Guidi A, Scilingo EP, Pietrini P, Bicchi A, Santello M, Ricciardi E. A synergy-based hand control is encoded in human motor cortical areas. eLife 2016; 5. [PMID: 26880543 PMCID: PMC4786436 DOI: 10.7554/elife.13420] [Citation(s) in RCA: 79] [Impact Index Per Article: 9.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/09/2015] [Accepted: 02/13/2016] [Indexed: 01/17/2023] Open
Abstract
How the human brain controls hand movements to carry out different tasks is still debated. The concept of synergy has been proposed to indicate functional modules that may simplify the control of hand postures by simultaneously recruiting sets of muscles and joints. However, whether and to what extent synergic hand postures are encoded as such at a cortical level remains unknown. Here, we combined kinematic, electromyography, and brain activity measures obtained by functional magnetic resonance imaging while subjects performed a variety of movements towards virtual objects. Hand postural information, encoded through kinematic synergies, were represented in cortical areas devoted to hand motor control and successfully discriminated individual grasping movements, significantly outperforming alternative somatotopic or muscle-based models. Importantly, hand postural synergies were predicted by neural activation patterns within primary motor cortex. These findings support a novel cortical organization for hand movement control and open potential applications for brain-computer interfaces and neuroprostheses. DOI:http://dx.doi.org/10.7554/eLife.13420.001 The human hand can perform an enormous range of movements with great dexterity. Some common everyday actions, such as grasping a coffee cup, involve the coordinated movement of all four fingers and thumb. Others, such as typing, rely on the ability of individual fingers to move relatively independently of one another. This flexibility is possible in part because of the complex anatomy of the hand, with its 27 bones and their connecting joints and muscles. But with this complexity comes a huge number of possibilities. Any movement-related task – such as picking up a cup – can be achieved via many different combinations of muscle contractions and joint positions. So how does the brain decide which muscles and joints to use? One theory is that the brain simplifies this problem by encoding particularly useful patterns of joint movements as distinct units or “synergies”. A given task can then be performed by selecting from a small number of synergies, avoiding the need to choose between huge numbers of options every time movement is required. Leo et al. now provide the first direct evidence for the encoding of synergies by the human brain. Volunteers lying inside a brain scanner reached towards virtual objects – from tennis rackets to toothpicks – while activity was recorded from the area of the brain that controls hand movements. As predicted, the scans showed specific and reproducible patterns of activity. Analysing these patterns revealed that each corresponded to a particular combination of joint positions. These activity patterns, or synergies, could even be ‘decoded’ to work out which type of movement a volunteer had just performed. Future experiments should examine how the brain combines synergies with sensory feedback to allow movements to be adjusted as they occur. Such findings could help to develop brain-computer interfaces and systems for controlling the movement of artificial limbs. DOI:http://dx.doi.org/10.7554/eLife.13420.002
Collapse
Affiliation(s)
- Andrea Leo
- Laboratory of Clinical Biochemistry and Molecular Biology, University of Pisa, Pisa, Italy.,Research Center 'E. Piaggio', University of Pisa, Pisa, Italy
| | - Giacomo Handjaras
- Laboratory of Clinical Biochemistry and Molecular Biology, University of Pisa, Pisa, Italy
| | - Matteo Bianchi
- Research Center 'E. Piaggio', University of Pisa, Pisa, Italy.,Advanced Robotics Department, Istituto Italiano di Tecnologia, Genova, Italy
| | - Hamal Marino
- Research Center 'E. Piaggio', University of Pisa, Pisa, Italy
| | - Marco Gabiccini
- Research Center 'E. Piaggio', University of Pisa, Pisa, Italy.,Advanced Robotics Department, Istituto Italiano di Tecnologia, Genova, Italy.,Department of Civil and Industrial Engineering, University of Pisa, Pisa, Italy
| | - Andrea Guidi
- Research Center 'E. Piaggio', University of Pisa, Pisa, Italy
| | - Enzo Pasquale Scilingo
- Research Center 'E. Piaggio', University of Pisa, Pisa, Italy.,Department of Information Engineering, University of Pisa, Pisa, Italy
| | - Pietro Pietrini
- Laboratory of Clinical Biochemistry and Molecular Biology, University of Pisa, Pisa, Italy.,Research Center 'E. Piaggio', University of Pisa, Pisa, Italy.,Clinical Psychology Branch, Pisa University Hospital, Pisa, Italy.,IMT School for Advanced Studies Lucca, Lucca, Italy
| | - Antonio Bicchi
- Research Center 'E. Piaggio', University of Pisa, Pisa, Italy.,Advanced Robotics Department, Istituto Italiano di Tecnologia, Genova, Italy
| | - Marco Santello
- School of Biological and Health Systems Engineering, Arizona State University, Tempe, United States
| | - Emiliano Ricciardi
- Laboratory of Clinical Biochemistry and Molecular Biology, University of Pisa, Pisa, Italy.,Research Center 'E. Piaggio', University of Pisa, Pisa, Italy
| |
Collapse
|
23
|
Santello M, Bianchi M, Gabiccini M, Ricciardi E, Salvietti G, Prattichizzo D, Ernst M, Moscatelli A, Jörntell H, Kappers AML, Kyriakopoulos K, Albu-Schäffer A, Castellini C, Bicchi A. Hand synergies: Integration of robotics and neuroscience for understanding the control of biological and artificial hands. Phys Life Rev 2016; 17:1-23. [PMID: 26923030 DOI: 10.1016/j.plrev.2016.02.001] [Citation(s) in RCA: 109] [Impact Index Per Article: 13.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/02/2016] [Accepted: 02/02/2016] [Indexed: 12/30/2022]
Abstract
The term 'synergy' - from the Greek synergia - means 'working together'. The concept of multiple elements working together towards a common goal has been extensively used in neuroscience to develop theoretical frameworks, experimental approaches, and analytical techniques to understand neural control of movement, and for applications for neuro-rehabilitation. In the past decade, roboticists have successfully applied the framework of synergies to create novel design and control concepts for artificial hands, i.e., robotic hands and prostheses. At the same time, robotic research on the sensorimotor integration underlying the control and sensing of artificial hands has inspired new research approaches in neuroscience, and has provided useful instruments for novel experiments. The ambitious goal of integrating expertise and research approaches in robotics and neuroscience to study the properties and applications of the concept of synergies is generating a number of multidisciplinary cooperative projects, among which the recently finished 4-year European project "The Hand Embodied" (THE). This paper reviews the main insights provided by this framework. Specifically, we provide an overview of neuroscientific bases of hand synergies and introduce how robotics has leveraged the insights from neuroscience for innovative design in hardware and controllers for biomedical engineering applications, including myoelectric hand prostheses, devices for haptics research, and wearable sensing of human hand kinematics. The review also emphasizes how this multidisciplinary collaboration has generated new ways to conceptualize a synergy-based approach for robotics, and provides guidelines and principles for analyzing human behavior and synthesizing artificial robotic systems based on a theory of synergies.
Collapse
Affiliation(s)
- Marco Santello
- School of Biological and Health Systems Engineering, Arizona State University, Tempe, AZ, USA.
| | - Matteo Bianchi
- Research Center 'E. Piaggio', University of Pisa, Pisa, Italy; Advanced Robotics Department, Istituto Italiano di Tecnologia (IIT), Genova, Italy
| | - Marco Gabiccini
- Research Center 'E. Piaggio', University of Pisa, Pisa, Italy; Advanced Robotics Department, Istituto Italiano di Tecnologia (IIT), Genova, Italy; Department of Civil and Industrial Engineering, University of Pisa, Pisa, Italy
| | - Emiliano Ricciardi
- Molecular Mind Laboratory, Dept. Surgical, Medical, Molecular Pathology and Critical Care, University of Pisa, Pisa, Italy; Research Center 'E. Piaggio', University of Pisa, Pisa, Italy
| | - Gionata Salvietti
- Department of Information Engineering and Mathematics, University of Siena, Siena, Italy
| | - Domenico Prattichizzo
- Department of Information Engineering and Mathematics, University of Siena, Siena, Italy; Advanced Robotics Department, Istituto Italiano di Tecnologia (IIT), Genova, Italy
| | - Marc Ernst
- Department of Cognitive Neuroscience and CITEC, Bielefeld University, Bielefeld, Germany
| | - Alessandro Moscatelli
- Department of Cognitive Neuroscience and CITEC, Bielefeld University, Bielefeld, Germany; Department of Systems Medicine and Centre of Space Bio-Medicine, Università di Roma "Tor Vergata", 00173, Rome, Italy
| | - Henrik Jörntell
- Neural Basis of Sensorimotor Control, Department of Experimental Medical Science, Lund University, Lund, Sweden
| | | | - Kostas Kyriakopoulos
- School of Mechanical Engineering, National Technical University of Athens, Greece
| | - Alin Albu-Schäffer
- DLR - German Aerospace Center, Institute of Robotics and Mechatronics, Oberpfaffenhofen, Germany
| | - Claudio Castellini
- DLR - German Aerospace Center, Institute of Robotics and Mechatronics, Oberpfaffenhofen, Germany
| | - Antonio Bicchi
- Research Center 'E. Piaggio', University of Pisa, Pisa, Italy; Advanced Robotics Department, Istituto Italiano di Tecnologia (IIT), Genova, Italy.
| |
Collapse
|
24
|
Bonaiuto J, Arbib MA. Learning to grasp and extract affordances: the Integrated Learning of Grasps and Affordances (ILGA) model. BIOLOGICAL CYBERNETICS 2015; 109:639-69. [PMID: 26585965 PMCID: PMC4656720 DOI: 10.1007/s00422-015-0666-2] [Citation(s) in RCA: 8] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 06/17/2015] [Accepted: 10/29/2015] [Indexed: 06/05/2023]
Abstract
The activity of certain parietal neurons has been interpreted as encoding affordances (directly perceivable opportunities) for grasping. Separate computational models have been developed for infant grasp learning and affordance learning, but no single model has yet combined these processes in a neurobiologically plausible way. We present the Integrated Learning of Grasps and Affordances (ILGA) model that simultaneously learns grasp affordances from visual object features and motor parameters for planning grasps using trial-and-error reinforcement learning. As in the Infant Learning to Grasp Model, we model a stage of infant development prior to the onset of sophisticated visual processing of hand-object relations, but we assume that certain premotor neurons activate neural populations in primary motor cortex that synergistically control different combinations of fingers. The ILGA model is able to extract affordance representations from visual object features, learn motor parameters for generating stable grasps, and generalize its learned representations to novel objects.
Collapse
Affiliation(s)
- James Bonaiuto
- Sobell Department of Motor Neuroscience and Movement Disorders, University College London, London, WC1N3BG, UK.
- Neuroscience Program, University of Southern California, Los Angeles, CA, 90089-2520, USA.
- USC Brain Project, University of Southern California, Los Angeles, CA, 90089-2520, USA.
| | - Michael A Arbib
- Neuroscience Program, University of Southern California, Los Angeles, CA, 90089-2520, USA
- USC Brain Project, University of Southern California, Los Angeles, CA, 90089-2520, USA
- Computer Science Department, University of Southern California, Los Angeles, CA, 90089-2520, USA
| |
Collapse
|
25
|
Rouse AG, Schieber MH. Spatiotemporal distribution of location and object effects in reach-to-grasp kinematics. J Neurophysiol 2015; 114:3268-82. [PMID: 26445870 DOI: 10.1152/jn.00686.2015] [Citation(s) in RCA: 15] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/09/2015] [Accepted: 10/01/2015] [Indexed: 02/06/2023] Open
Abstract
In reaching to grasp an object, the arm transports the hand to the intended location as the hand shapes to grasp the object. Prior studies that tracked arm endpoint and grip aperture have shown that reaching and grasping, while proceeding in parallel, are interdependent to some degree. Other studies of reaching and grasping that have examined the joint angles of all five digits as the hand shapes to grasp various objects have not tracked the joint angles of the arm as well. We, therefore, examined 22 joint angles from the shoulder to the five digits as monkeys reached, grasped, and manipulated in a task that dissociated location and object. We quantified the extent to which each angle varied depending on location, on object, and on their interaction, all as a function of time. Although joint angles varied depending on both location and object beginning early in the movement, an early phase of location effects in joint angles from the shoulder to the digits was followed by a later phase in which object effects predominated at all joint angles distal to the shoulder. Interaction effects were relatively small throughout the reach-to-grasp. Whereas reach trajectory was influenced substantially by the object, grasp shape was comparatively invariant to location. Our observations suggest that neural control of reach-to-grasp may occur largely in two sequential phases: the first determining the location to which the arm transports the hand, and the second shaping the entire upper extremity to grasp and manipulate the object.
Collapse
Affiliation(s)
- Adam G Rouse
- Department of Neurology, University of Rochester, Rochester, New York; Department of Neurobiology and Anatomy, University of Rochester, Rochester, New York; and Department of Biomedical Engineering, University of Rochester, Rochester, New York
| | - Marc H Schieber
- Department of Neurology, University of Rochester, Rochester, New York; Department of Neurobiology and Anatomy, University of Rochester, Rochester, New York; and Department of Biomedical Engineering, University of Rochester, Rochester, New York
| |
Collapse
|
26
|
Hand use predicts the structure of representations in sensorimotor cortex. Nat Neurosci 2015; 18:1034-40. [PMID: 26030847 DOI: 10.1038/nn.4038] [Citation(s) in RCA: 140] [Impact Index Per Article: 15.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/31/2015] [Accepted: 05/07/2015] [Indexed: 11/08/2022]
Abstract
Fine finger movements are controlled by the population activity of neurons in the hand area of primary motor cortex. Experiments using microstimulation and single-neuron electrophysiology suggest that this area represents coordinated multi-joint, rather than single-finger movements. However, the principle by which these representations are organized remains unclear. We analyzed activity patterns during individuated finger movements using functional magnetic resonance imaging (fMRI). Although the spatial layout of finger-specific activity patterns was variable across participants, the relative similarity between any pair of activity patterns was well preserved. This invariant organization was better explained by the correlation structure of everyday hand movements than by correlated muscle activity. This also generalized to an experiment using complex multi-finger movements. Finally, the organizational structure correlated with patterns of involuntary co-contracted finger movements for high-force presses. Together, our results suggest that hand use shapes the relative arrangement of finger-specific activity patterns in sensory-motor cortex.
Collapse
|
27
|
Tagliabue M, Ciancio AL, Brochier T, Eskiizmirliler S, Maier MA. Differences between kinematic synergies and muscle synergies during two-digit grasping. Front Hum Neurosci 2015; 9:165. [PMID: 25859208 PMCID: PMC4374551 DOI: 10.3389/fnhum.2015.00165] [Citation(s) in RCA: 39] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/19/2014] [Accepted: 03/10/2015] [Indexed: 12/19/2022] Open
Abstract
The large number of mechanical degrees of freedom of the hand is not fully exploited during actual movements such as grasping. Usually, angular movements in various joints tend to be coupled, and EMG activities in different hand muscles tend to be correlated. The occurrence of covariation in the former was termed kinematic synergies, in the latter muscle synergies. This study addresses two questions: (i) Whether kinematic and muscle synergies can simultaneously accommodate for kinematic and kinetic constraints. (ii) If so, whether there is an interrelation between kinematic and muscle synergies. We used a reach-grasp-and-pull paradigm and recorded the hand kinematics as well as eight surface EMGs. Subjects had to either perform a precision grip or side grip and had to modify their grip force in order to displace an object against a low or high load. The analysis was subdivided into three epochs: reach, grasp-and-pull, and static hold. Principal component analysis (PCA, temporal or static) was performed separately for all three epochs, in the kinematic and in the EMG domain. PCA revealed that (i) Kinematic- and muscle-synergies can simultaneously accommodate kinematic (grip type) and kinetic task constraints (load condition). (ii) Upcoming grip and load conditions of the grasp are represented in kinematic- and muscle-synergies already during reach. Phase plane plots of the principal muscle-synergy against the principal kinematic synergy revealed (iii) that the muscle-synergy is linked (correlated, and in phase advance) to the kinematic synergy during reach and during grasp-and-pull. Furthermore (iv), pair-wise correlations of EMGs during hold suggest that muscle-synergies are (in part) implemented by coactivation of muscles through common input. Together, these results suggest that kinematic synergies have (at least in part) their origin not just in muscular activation, but in synergistic muscle activation. In short: kinematic synergies may result from muscle synergies.
Collapse
Affiliation(s)
- Michele Tagliabue
- Neuroscience Research Federation FR3636, CNRS, Université Paris Descartes Paris, France ; Centre de Neurophysique, Physiologie et Pathologie, UMR 8119, CNRS, Université Paris Descartes Sorbonne Paris Cité, Paris, France
| | - Anna Lisa Ciancio
- Laboratory of Biomedical Robotic and Biomicrosystem, Università Campus Bio-Medico di Roma Roma, Italy
| | - Thomas Brochier
- Institut de Neurosciences de la Timone, UMR 7289, CNRS, Aix-Marseille Université Marseille, France
| | - Selim Eskiizmirliler
- Neuroscience Research Federation FR3636, CNRS, Université Paris Descartes Paris, France ; Life Sciences Department, Université Paris Diderot Sorbonne Paris Cité, Paris, France
| | - Marc A Maier
- Neuroscience Research Federation FR3636, CNRS, Université Paris Descartes Paris, France ; Life Sciences Department, Université Paris Diderot Sorbonne Paris Cité, Paris, France
| |
Collapse
|