1
|
Takazawa T, Suzuki Y, Nakamura A, Matsuo R, Morasso P, Nomura T. How the brain can be trained to achieve an intermittent control strategy for stabilizing quiet stance by means of reinforcement learning. BIOLOGICAL CYBERNETICS 2024; 118:229-248. [PMID: 38995347 PMCID: PMC11289178 DOI: 10.1007/s00422-024-00993-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/08/2024] [Accepted: 06/21/2024] [Indexed: 07/13/2024]
Abstract
The stabilization of human quiet stance is achieved by a combination of the intrinsic elastic properties of ankle muscles and an active closed-loop activation of the ankle muscles, driven by the delayed feedback of the ongoing sway angle and the corresponding angular velocity in a way of a delayed proportional (P) and derivative (D) feedback controller. It has been shown that the active component of the stabilization process is likely to operate in an intermittent manner rather than as a continuous controller: the switching policy is defined in the phase-plane, which is divided in dangerous and safe regions, separated by appropriate switching boundaries. When the state enters a dangerous region, the delayed PD control is activated, and it is switched off when it enters a safe region, leaving the system to evolve freely. In comparison with continuous feedback control, the intermittent mechanism is more robust and capable to better reproduce postural sway patterns in healthy people. However, the superior performance of the intermittent control paradigm as well as its biological plausibility, suggested by experimental evidence of the intermittent activation of the ankle muscles, leaves open the quest of a feasible learning process, by which the brain can identify the appropriate state-dependent switching policy and tune accordingly the P and D parameters. In this work, it is shown how such a goal can be achieved with a reinforcement motor learning paradigm, building upon the evidence that, in general, the basal ganglia are known to play a central role in reinforcement learning for action selection and, in particular, were found to be specifically involved in postural stabilization.
Collapse
Affiliation(s)
- Tomoki Takazawa
- Graduate School of Engineering Science, Osaka University, 1-3 Machikaneyama, Toyonaka, Osaka, 5608531, Japan
| | - Yasuyuki Suzuki
- Graduate School of Engineering Science, Osaka University, 1-3 Machikaneyama, Toyonaka, Osaka, 5608531, Japan
| | - Akihiro Nakamura
- Graduate School of Engineering Science, Osaka University, 1-3 Machikaneyama, Toyonaka, Osaka, 5608531, Japan
| | - Risa Matsuo
- Graduate School of Engineering Science, Osaka University, 1-3 Machikaneyama, Toyonaka, Osaka, 5608531, Japan
| | - Pietro Morasso
- Istituto Italiano di Tecnologia, Via Enrico Melen 83, Bldg B, 16152, Genoa, Italy
| | - Taishin Nomura
- Graduate School of Engineering Science, Osaka University, 1-3 Machikaneyama, Toyonaka, Osaka, 5608531, Japan.
- Graduate School of Informatics, Kyoto University, Yoshida Honmachi, Sakyo-ku, Kyoto, 6068501, Japan.
| |
Collapse
|
2
|
Suzuki Y, Nakamura A, Milosevic M, Nomura K, Tanahashi T, Endo T, Sakoda S, Morasso P, Nomura T. Postural instability via a loss of intermittent control in elderly and patients with Parkinson's disease: A model-based and data-driven approach. CHAOS (WOODBURY, N.Y.) 2020; 30:113140. [PMID: 33261318 DOI: 10.1063/5.0022319] [Citation(s) in RCA: 16] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/21/2020] [Accepted: 10/28/2020] [Indexed: 06/12/2023]
Abstract
Postural instability is one of the major symptoms of Parkinson's disease. Here, we assimilated a model of intermittent delay feedback control during quiet standing into postural sway data from healthy young and elderly individuals as well as patients with Parkinson's disease to elucidate the possible mechanisms of instability. Specifically, we estimated the joint probability distribution of a set of parameters in the model using the Bayesian parameter inference such that the model with the inferred parameters can best-fit sway data for each individual. It was expected that the parameter values for three populations would distribute differently in the parameter space depending on their balance capability. Because the intermittent control model is parameterized by a parameter associated with the degree of intermittency in the control, it can represent not only the intermittent model but also the traditional continuous control model with no intermittency. We showed that the inferred parameter values for the three groups of individuals are classified into two major groups in the parameter space: one represents the intermittent control mostly for healthy people and patients with mild postural symptoms and the other the continuous control mostly for some elderly and patients with severe postural symptoms. The results of this study may be interpreted by postulating that increased postural instability in most Parkinson's patients and some elderly persons might be characterized as a dynamical disease.
Collapse
Affiliation(s)
- Yasuyuki Suzuki
- Graduate School of Engineering Science, Osaka University, Osaka 5608531, Japan
| | - Akihiro Nakamura
- Graduate School of Engineering Science, Osaka University, Osaka 5608531, Japan
| | - Matija Milosevic
- Graduate School of Engineering Science, Osaka University, Osaka 5608531, Japan
| | - Kunihiko Nomura
- Department of Information Technology and Social Sciences, Osaka University of Economics, Osaka 5338533, Japan
| | - Takao Tanahashi
- Department of Neurology, Osaka Rosai Hospital, Osaka 5918025, Japan
| | - Takuyuki Endo
- Department of Neurology, Osaka Toneyama Medical Center, Osaka 5608552, Japan
| | - Saburo Sakoda
- Department of Neurology, Osaka Toneyama Medical Center, Osaka 5608552, Japan
| | - Pietro Morasso
- Center for Human Technologies, Istituto Italiano di Tecnologia, Genoa 16163, Italy
| | - Taishin Nomura
- Graduate School of Engineering Science, Osaka University, Osaka 5608531, Japan
| |
Collapse
|