Internal models in sensorimotor integration: perspectives from adaptive control theory

Learning to use a brain-machine interface: model, simulation and analysis

This paper presents a model of the learning process occurring during operation of a closed-loop brain-machine interface. The model consists of a population of simulated cortical neurons, a decoder that transforms neural activity into motor output, a feedback controller whose role is to reduce the error based on an error-descent algorithm, and an open-loop controller whose parameters are updated based on the corrections made by the feedback controller. We present evidence of the convergence of the internal model to the decoder's inverse model and use global sensitivity analysis to study the convergence's dependence on the parameters of the overall learning model. This model can be used as a simulation tool that predicts the outcome of closed-loop BMI experiments.

Computational approaches to spatial orientation: from transfer functions to dynamic Bayesian inference

Spatial orientation is the sense of body orientation and self-motion relative to the stationary environment, fundamental to normal waking behavior and control of everyday motor actions including eye movements, postural control, and locomotion. The brain achieves spatial orientation by integrating visual, vestibular, and somatosensory signals. Over the past years, considerable progress has been made toward understanding how these signals are processed by the brain using multiple computational approaches that include frequency domain analysis, the concept of internal models, observer theory, Bayesian theory, and Kalman filtering. Here we put these approaches in context by examining the specific questions that can be addressed by each technique and some of the scientific insights that have resulted. We conclude with a recent application of particle filtering, a probabilistic simulation technique that aims to generate the most likely state estimates by incorporating internal models of sensor dynamics and physical laws and noise associated with sensory processing as well as prior knowledge or experience. In this framework, priors for low angular velocity and linear acceleration can explain the phenomena of velocity storage and frequency segregation, both of which have been modeled previously using arbitrary low-pass filtering. How Kalman and particle filters may be implemented by the brain is an emerging field. Unlike past neurophysiological research that has aimed to characterize mean responses of single neurons, investigations of dynamic Bayesian inference should attempt to characterize population activities that constitute probabilistic representations of sensory and prior information.

Asymmetric adaptation with functional advantage in human sensorimotor control

Human movement control is inherently stochastic, requiring continuous estimation of self-motion based upon noisy sensory inputs. The nervous system must determine which sensory signals are relevant on a time scale that enables successful behavior. In human stance control, failure to effectively adapt to changing sensory contexts could lead to injurious falls. Nonlinear changes in postural sway amplitude in response to changes in sensory environmental motion have indicated a dynamic changing of the weighting of the nervous system's multiple sensory inputs so that estimates are based upon the most relevant and accurate information available. However, the time scale of these changes is virtually unknown. Results here show systematic changes in postural gain when visual scene motion amplitude is increased or decreased abruptly, consistent with sensory re-weighting. However, this re-weighting displayed a temporal asymmetry. When visual motion increased, gain decreased within 5 s to a value near its asymptotic value. In contrast, when visual motion decreased, it took an additional 5 s for gain to increase by a similar absolute amount. Suddenly increasing visual motion amplitude threatens balance if gain remains high, and rapid down-weighting of the sensory signal is required to avoid falling. By contrast, slow up-weighting suggests a conservative CNS strategy. It may not be functional to rapidly up-weight with transient changes in the sensory environment. Only sustained changes necessitate the slower up-weighting process. Such results add to our understanding of adaptive processing, identifying a temporal asymmetry in sensory re-weighting dynamics that could be a general property of adaptive estimation in the nervous system.

Modeling sensorimotor control of human upright stance

We model human postural control of upright stance during external disturbances and voluntary lean. Our focus is on how data from various sensors are combined to estimate these disturbances. Whereas most current engineering models of multisensory estimation rely on "internal observers" and complex processing, we compute our estimates by simple sensor fusion mechanisms, i.e., weighted sums of sensory signals combined with thresholds. We show with simulations that this simple device mimics human-like postural behavior in a wide range of situations and diseases. We have now embodied our mechanism in a biped humanoid robot to show that it works in the real world with complex, noisy, and imperfectly known sensors and effectors. On the other hand, we find that the more complex, internal-observer approach, when applied to bipedal posture, can also yield human-like behavior. We suggest that humans use both mechanisms: simple, fast sensor fusions with thresholding for automatic reactions (default mechanism), and more complex methods for voluntary movements. We suggest also that the fusion with thresholding mechanisms are optimized during phylogenesis but are mainly hardwired in any one organism, whereas sensorimotor learning and optimization is mainly a domain of the internal observers.

Auditory-motor mapping for pitch control in singers and nonsingers

Little is known about the basic processes underlying the behavior of singing. This experiment was designed to examine differences in the representation of the mapping between fundamental frequency (F0) feedback and the vocal production system in singers and nonsingers. Auditory feedback regarding F0 was shifted down in frequency while participants sang the consonant-vowel /ta/. During the initial frequency-altered trials, singers compensated to a lesser degree than nonsingers, but this difference was reduced with continued exposure to frequency-altered feedback. After brief exposure to frequency altered auditory feedback, both singers and nonsingers suddenly heard their F0 unaltered. When participants received this unaltered feedback, only singers' F0 values were found to be significantly higher than their F0 values produced during baseline and control trials. These aftereffects in singers were replicated when participants sang a different note than the note they produced while hearing altered feedback. Together, these results suggest that singers rely more on internal models than nonsingers to regulate vocal productions rather than real time auditory feedback.

A general internal model approach for motion learning

In this paper, we present a general internal model (GIM) approach for motion skill learning at elementary and coordination levels. A unified internal model (IM) is developed for describing discrete and rhythmic movements. Through analysis, we show that the GIM possesses temporal and spatial scalabilities which are defined as the ability to generate similar movement patterns directly by means of tuning some parameters of the IM. With scalability, the learning or training process can be avoided when dealing with similar tasks. The coordination is implemented in the GIM with appropriate phase shifts among multiple IMs under an overall architecture. To facilitate the establishment of the GIM, in this paper, we further explored algorithms for detecting periodicity of and phase difference between rhythmic movements, and neural network structures suitable for learning motion patterns. Through three illustrative examples, we show that the human behavior patterns with single or multiple limbs can be easily learned and established by the GIM at the elementary and coordination levels.

Homeostasis of exercise hyperpnea and optimal sensorimotor integration: the internal model paradigm

Homeostasis is a basic tenet of biomedicine and an open problem for many physiological control systems. Among them, none has been more extensively studied and intensely debated than the dilemma of exercise hyperpnea - a paradoxical homeostatic increase of respiratory ventilation that is geared to metabolic demands instead of the normal chemoreflex mechanism. Classical control theory has led to a plethora of "feedback/feedforward control" or "set point" hypotheses for homeostatic regulation, yet so far none of them has proved satisfactory in explaining exercise hyperpnea and its interactions with other respiratory inputs. Instead, the available evidence points to a far more sophisticated respiratory controller capable of integrating multiple afferent and efferent signals in adapting the ventilatory pattern toward optimality relative to conflicting homeostatic, energetic and other objectives. This optimality principle parsimoniously mimics exercise hyperpnea, chemoreflex and a host of characteristic respiratory responses to abnormal gas exchange or mechanical loading/unloading in health and in cardiopulmonary diseases - all without resorting to a feedforward "exercise stimulus". Rather, an emergent controller signal encoding the projected metabolic level is predicted by the principle as an exercise-induced 'mental percept' or 'internal model', presumably engendered by associative learning (operant conditioning or classical conditioning) which achieves optimality through continuous identification of, and adaptation to, the causal relationship between respiratory motor output and resultant chemical-mechanical afferent feedbacks. This internal model self-tuning adaptive control paradigm opens a new challenge and exciting opportunity for experimental and theoretical elucidations of the mechanisms of respiratory control - and of homeostatic regulation and sensorimotor integration in general.

Speech production: the force of your words

Research on speech production has traditionally focused on how acoustic goals are met. A recent study shows that talking also involves somatosensory goals that do not have any acoustic consequences.

Nonassociative learning as gated neural integrator and differentiator in stimulus-response pathways

Nonassociative learning is a basic neuroadaptive behavior exhibited across animal phyla and sensory modalities but its role in brain intelligence is unclear. Current literature on habituation and sensitization, the classic "dual process" of nonassociative learning, gives highly incongruous accounts between varying experimental paradigms. Here we propose a general theory of nonassociative learning featuring four base modes: habituation/primary sensitization in primary stimulus-response pathways, and desensitization/secondary sensitization in secondary stimulus-response pathways. Primary and secondary modes of nonassociative learning are distinguished by corresponding activity-dependent recall, or nonassociative gating, of neurotransmission memory. From the perspective of brain computation, nonassociative learning is a form of integral-differential calculus whereas nonassociative gating is a form of Boolean logic operator--both dynamically transforming the stimulus-response relationship. From the perspective of sensory integration, nonassociative gating provides temporal filtering whereas nonassociative learning affords low-pass, high-pass or band-pass/band-stop frequency filtering--effectively creating an intelligent sensory firewall that screens all stimuli for attention and resultant internal model adaptation and reaction. This unified framework ties together many salient characteristics of nonassociative learning and nonassociative gating and suggests a common kernel that correlates with a wide variety of sensorimotor integration behaviors such as central resetting and self-organization of sensory inputs, fail-safe sensorimotor compensation, integral-differential and gated modulation of sensorimotor feedbacks, alarm reaction, novelty detection and selective attention, as well as a variety of mental and neurological disorders such as sensorimotor instability, attention deficit hyperactivity, sensory defensiveness, autism, nonassociative fear and anxiety, schizophrenia, addiction and craving, pain sensitization and phantom sensations, etc.

Adaptive control of vowel formant frequency: evidence from real-time formant manipulation

Auditory feedback during speech production is known to play a role in speech sound acquisition and is also important for the maintenance of accurate articulation. In two studies the first formant (F1) of monosyllabic consonant-vowel-consonant words (CVCs) was shifted electronically and fed back to the participant very quickly so that participants perceived the modified speech as their own productions. When feedback was shifted up (experiment 1 and 2) or down (experiment 1) participants compensated by producing F1 in the opposite frequency direction from baseline. The threshold size of manipulation that initiated a compensation in F1 was usually greater than 60 Hz. When normal feedback was returned, F1 did not return immediately to baseline but showed an exponential deadaptation pattern. Experiment 1 showed that this effect was not influenced by the direction of the F1 shift, with both raising and lowering of F1 exhibiting the same effects. Experiment 2 showed that manipulating the number of trials that F1 was held at the maximum shift in frequency (0, 15, 45 trials) did not influence the recovery from adaptation. There was a correlation between the lag-one autocorrelation of trial-to-trial changes in F1 in the baseline recordings and the magnitude of compensation. Some participants therefore appeared to more actively stabilize their productions from trial-to-trial. The results provide insight into the perceptual control of speech and the representations that govern sensorimotor coordination.