1
|
Roth AM, Lokesh R, Tang J, Buggeln JH, Smith C, Calalo JA, Sullivan SR, Ngo T, Germain LS, Carter MJ, Cashaback JGA. Punishment Leads to Greater Sensorimotor Learning But Less Movement Variability Compared to Reward. Neuroscience 2024; 540:12-26. [PMID: 38220127 PMCID: PMC10922623 DOI: 10.1016/j.neuroscience.2024.01.004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/18/2023] [Revised: 01/05/2024] [Accepted: 01/09/2024] [Indexed: 01/16/2024]
Abstract
When a musician practices a new song, hitting a correct note sounds pleasant while striking an incorrect note sounds unpleasant. Such reward and punishment feedback has been shown to differentially influence the ability to learn a new motor skill. Recent work has suggested that punishment leads to greater movement variability, which causes greater exploration and faster learning. To further test this idea, we collected 102 participants over two experiments. Unlike previous work, in Experiment 1 we found that punishment did not lead to faster learning compared to reward (n = 68), but did lead to a greater extent of learning. Surprisingly, we also found evidence to suggest that punishment led to less movement variability, which was related to the extent of learning. We then designed a second experiment that did not involve adaptation, allowing us to further isolate the influence of punishment feedback on movement variability. In Experiment 2, we again found that punishment led to significantly less movement variability compared to reward (n = 34). Collectively our results suggest that punishment feedback leads to less movement variability. Future work should investigate whether punishment feedback leads to a greater knowledge of movement variability and or increases the sensitivity of updating motor actions.
Collapse
Affiliation(s)
- Adam M Roth
- Department of Mechanical Engineering, University of Delaware, United States
| | - Rakshith Lokesh
- Department of Biomedical Engineering, University of Delaware, United States
| | - Jiaqiao Tang
- Department of Kinesiology, McMaster University, Canada
| | - John H Buggeln
- Department of Biomedical Engineering, University of Delaware, United States
| | - Carly Smith
- Department of Biomedical Engineering, University of Delaware, United States
| | - Jan A Calalo
- Department of Mechanical Engineering, University of Delaware, United States
| | - Seth R Sullivan
- Department of Biomedical Engineering, University of Delaware, United States
| | - Truc Ngo
- Department of Biomedical Engineering, University of Delaware, United States
| | | | | | - Joshua G A Cashaback
- Department of Mechanical Engineering, University of Delaware, United States; Department of Biomedical Engineering, University of Delaware, United States; Kinesiology and Applied Physiology, University of Delaware, United States; Interdisciplinary Neuroscience Graduate Program, University of Delaware, United States; Biomechanics and Movement Science Program, University of Delaware, United States; Department of Kinesiology, McMaster University, Canada.
| |
Collapse
|
2
|
Roth AM, Calalo JA, Lokesh R, Sullivan SR, Grill S, Jeka JJ, van der Kooij K, Carter MJ, Cashaback JGA. Reinforcement-based processes actively regulate motor exploration along redundant solution manifolds. Proc Biol Sci 2023; 290:20231475. [PMID: 37848061 PMCID: PMC10581769 DOI: 10.1098/rspb.2023.1475] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/29/2023] [Accepted: 09/06/2023] [Indexed: 10/19/2023] Open
Abstract
From a baby's babbling to a songbird practising a new tune, exploration is critical to motor learning. A hallmark of exploration is the emergence of random walk behaviour along solution manifolds, where successive motor actions are not independent but rather become serially dependent. Such exploratory random walk behaviour is ubiquitous across species' neural firing, gait patterns and reaching behaviour. The past work has suggested that exploratory random walk behaviour arises from an accumulation of movement variability and a lack of error-based corrections. Here, we test a fundamentally different idea-that reinforcement-based processes regulate random walk behaviour to promote continual motor exploration to maximize success. Across three human reaching experiments, we manipulated the size of both the visually displayed target and an unseen reward zone, as well as the probability of reinforcement feedback. Our empirical and modelling results parsimoniously support the notion that exploratory random walk behaviour emerges by utilizing knowledge of movement variability to update intended reach aim towards recently reinforced motor actions. This mechanism leads to active and continuous exploration of the solution manifold, currently thought by prominent theories to arise passively. The ability to continually explore muscle, joint and task redundant solution manifolds is beneficial while acting in uncertain environments, during motor development or when recovering from a neurological disorder to discover and learn new motor actions.
Collapse
Affiliation(s)
- Adam M. Roth
- Department of Mechanical Engineering, University of Delaware, Newark, DE 19716, USA
| | - Jan A. Calalo
- Department of Mechanical Engineering, University of Delaware, Newark, DE 19716, USA
| | - Rakshith Lokesh
- Department of Biomedical Engineering, University of Delaware, Newark, DE 19716, USA
| | - Seth R. Sullivan
- Department of Biomedical Engineering, University of Delaware, Newark, DE 19716, USA
| | - Stephen Grill
- Kinesiology and Applied Physiology, University of Delaware, Newark, DE 19716, USA
| | - John J. Jeka
- Kinesiology and Applied Physiology, University of Delaware, Newark, DE 19716, USA
- Interdisciplinary Neuroscience Graduate Program, University of Delaware, Newark, DE 19716, USA
- Biomechanics and Movement Science Program, University of Delaware, Newark, DE 19716, USA
| | - Katinka van der Kooij
- Faculty of Behavioural and Movement Science, Vrije University Amsterdam, Amsterdam, 1081HV, The Netherlands
| | - Michael J. Carter
- Department of Kinesiology, McMaster University, Room 203, Ivor Wynne Centre, Hamilton, L8S 4L8, Ontario, Canada
| | - Joshua G. A. Cashaback
- Department of Mechanical Engineering, University of Delaware, Newark, DE 19716, USA
- Department of Biomedical Engineering, University of Delaware, Newark, DE 19716, USA
- Kinesiology and Applied Physiology, University of Delaware, Newark, DE 19716, USA
- Interdisciplinary Neuroscience Graduate Program, University of Delaware, Newark, DE 19716, USA
- Biomechanics and Movement Science Program, University of Delaware, Newark, DE 19716, USA
| |
Collapse
|
3
|
Orientation control strategies and adaptation to a visuomotor perturbation in rotational hand movements. PLoS Comput Biol 2022; 18:e1010248. [DOI: 10.1371/journal.pcbi.1010248] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/26/2022] [Revised: 12/15/2022] [Accepted: 11/14/2022] [Indexed: 12/12/2022] Open
Abstract
Computational approaches to biological motor control are used to discover the building blocks of human motor behaviour. Models explaining features of human hand movements have been studied thoroughly, yet only a few studies attempted to explain the control of the orientation of the hand; instead, they mainly focus on the control of hand translation, predominantly in a single plane. In this study, we present a new methodology to study the way humans control the orientation of their hands in three dimensions and demonstrate it in two sequential experiments. We developed a quaternion-based score that quantifies the geodicity of rotational hand movements and evaluated it experimentally. In the first experiment, participants performed a simple orientation-matching task with a robotic manipulator. We found that rotations are generally performed by following a geodesic in the quaternion hypersphere, which suggests that, similarly to translation, the orientation of the hand is centrally controlled, possibly by optimizing geometrical properties of the hand’s rotation. This result established a baseline for the study of human response to perturbed visual feedback of the orientation of the hand. In the second experiment, we developed a novel visuomotor rotation task in which the rotation is applied on the hand’s rotation, and studied the adaptation of participants to this rotation, and the transfer of the adaptation to a different initial orientation. We observed partial adaptation to the rotation. The patterns of the transfer of the adaptation to a different initial orientation were consistent with the representation of the orientation in extrinsic coordinates. The methodology that we developed allows for studying the control of a rigid body without reducing the dimensionality of the task. The results of the two experiments open questions for future studies regarding the mechanisms underlying the central control of hand orientation. These results can be of benefit for many applications that involve fine manipulation of rigid bodies, such as teleoperation and neurorehabilitation.
Collapse
|
4
|
Specifying Inputs for the Computational Structure of a Surgical System via Optical Method and DLT Algorithm Based on In Vitro Experiments on Cardiovascular Tissue in Minimally Invasive and Robotic Surgery. SENSORS 2022; 22:s22062335. [PMID: 35336506 PMCID: PMC8955807 DOI: 10.3390/s22062335] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 12/26/2021] [Revised: 03/10/2022] [Accepted: 03/15/2022] [Indexed: 12/04/2022]
Abstract
With the application of four optical CMOS sensors, it was possible to capture the trajectory of an endoscopic tool during an in vitro surgical experiment on a cardiovascular preparation. This was due to the possibility of obtaining a path when a reflective marker was attached. In the work, APAS (Ariel Performance Analysis System) software and DLT (direct linear transformation) algorithm were used. This made it possible to acquire kinematic inputs to the computational model of dynamics, which enabled, regardless of the type of surgical robot structure, derivation of the analogous motion of an endoscopic effector due to the mathematical transformation of the trajectory to joints coordinates. Experiments were carried out with the participation of a practiced cardiac surgeon employing classic endoscopic instruments and robot surgical systems. The results indicated by the experiment showed that the inverse task of kinematics of position for the surgical robot with RCM (remote center of motion) structure was solved. The achieved results from the experiment were used as inputs for deriving a numerical dynamics model of surgical robot during transient states that was obtained by applying the finite element method and by driving dynamics moments acquired through the block diagrams method using a steering system with DC (direct current) motor and PID (proportional–integral–derivative) controller. The results section illustrates the course of kinematic values of endoscopic tools which were employed to apply numerical models as inputs, the course of the driving torque of the model of the surgical robot that enabled the selection of the drive system and the strength values, stresses and displacements according to von Mises hypothesis in its structure during the analysis of transient states that made it possible to establish the strength safety of the surgical robot. For the conducted experiments, the accuracy was ±2 [mm]. In the paper, the employment of optical CMOS sensors in surgical robotics and endoscopy is discussed. The paper concludes that the usage of optical sensors for determining inputs for numerical models of dynamics of surgical robots provides the basis for setting the course of physical quantities that appear in their real object structure, in manners close to reality.
Collapse
|
5
|
Zruya O, Sharon Y, Kossowsky H, Forni F, Geftler A, Nisky I. A New Power Law Linking the Speed to the Geometry of Tool-Tip Orientation in Teleoperation of a Robot-Assisted Surgical System. IEEE Robot Autom Lett 2022. [DOI: 10.1109/lra.2022.3193485] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Affiliation(s)
- Or Zruya
- Department of Biomedical Engineering and the Zlotowsky Center for Neuroscience, Ben-Gurion University of the Negev, Beer Sheva, Israel
| | - Yarden Sharon
- Department of Biomedical Engineering and the Zlotowsky Center for Neuroscience, Ben-Gurion University of the Negev, Beer Sheva, Israel
| | - Hanna Kossowsky
- Department of Biomedical Engineering and the Zlotowsky Center for Neuroscience, Ben-Gurion University of the Negev, Beer Sheva, Israel
| | - Fulvio Forni
- Department of Engineering, University of Cambridge, Cambridge, U.K
| | - Alex Geftler
- Department of Orthopedic Surgery, Soroka Medical Center, Ben-Gurion University of the Negev, Beer Sheva, Israel
| | - Ilana Nisky
- Department of Biomedical Engineering and the Zlotowsky Center for Neuroscience, Ben-Gurion University of the Negev, Beer Sheva, Israel
| |
Collapse
|
6
|
Bahar L, Sharon Y, Nisky I. Surgeon-Centered Analysis of Robot-Assisted Needle Driving Under Different Force Feedback Conditions. Front Neurorobot 2020; 13:108. [PMID: 32038218 PMCID: PMC6993204 DOI: 10.3389/fnbot.2019.00108] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/30/2019] [Accepted: 12/06/2019] [Indexed: 11/24/2022] Open
Abstract
Robotic assisted minimally invasive surgery (RAMIS) systems present many advantages to the surgeon and patient over open and standard laparoscopic surgery. However, haptic feedback, which is crucial for the success of many surgical procedures, is still an open challenge in RAMIS. Understanding the way that haptic feedback affects performance and learning can be useful in the development of haptic feedback algorithms and teleoperation control systems. In this study, we examined the performance and learning of inexperienced participants under different haptic feedback conditions in a task of surgical needle driving via a soft homogeneous deformable object-an artificial tissue. We designed an experimental setup to characterize their movement trajectories and the forces that they applied on the artificial tissue. Participants first performed the task in an open condition, with a standard surgical needle holder, followed by teleoperation in one of three feedback conditions: (1) no haptic feedback, (2) haptic feedback based on position exchange, and (3) haptic feedback based on direct recording from a force sensor, and then again with the open needle holder. To quantify the effect of different force feedback conditions on the quality of needle driving, we developed novel metrics that assess the kinematics of needle driving and the tissue interaction forces, and we combined our novel metrics with classical metrics. We analyzed the final teleoperated performance in each condition, the improvement during teleoperation, and the aftereffect of teleoperation on the performance when using the open needle driver. We found that there is no significant difference in the final performance and in the aftereffect between the 3 conditions. Only the two conditions with force feedback presented statistically significant improvement during teleoperation in several of the metrics, but when we compared directly between the improvements in the three different feedback conditions none of the effects reached statistical significance. We discuss possible explanations for the relative similarity in performance. We conclude that we developed several new metrics for the quality of surgical needle driving, but even with these detailed metrics, the advantage of state of the art force feedback methods to tasks that require interaction with homogeneous soft tissue is questionable.
Collapse
Affiliation(s)
| | | | - Ilana Nisky
- Department of Biomedical Engineering, Zlotowski Center of Neuroscience, Ben-Gurion University of the Negev, Be'er Sheva, Israel
| |
Collapse
|