A Hebbian feedback covariance learning paradigm for self-tuning optimal control

A neuro-inspired spike-based PID motor controller for multi-motor robots with low cost FPGAs

In this paper we present a neuro-inspired spike-based close-loop controller written in VHDL and implemented for FPGAs. This controller has been focused on controlling a DC motor speed, but only using spikes for information representation, processing and DC motor driving. It could be applied to other motors with proper driver adaptation. This controller architecture represents one of the latest layers in a Spiking Neural Network (SNN), which implements a bridge between robotics actuators and spike-based processing layers and sensors. The presented control system fuses actuation and sensors information as spikes streams, processing these spikes in hard real-time, implementing a massively parallel information processing system, through specialized spike-based circuits. This spike-based close-loop controller has been implemented into an AER platform, designed in our labs, that allows direct control of DC motors: the AER-Robot. Experimental results evidence the viability of the implementation of spike-based controllers, and hardware synthesis denotes low hardware requirements that allow replicating this controller in a high number of parallel controllers working together to allow a real-time robot control.

STUDY OF HUMAN THERMOREGULATION: ADAPTIVE OPTIMIZATION CONTROL THEORY ANALYSIS

An example of homeostasis is temperature regulation at a desired level; this physiological process leads to the preservation of a stable biological environment. A control-theory–based model permits a biomedical engineer to understand the complex operation of thermoregulation, by converting general information to knowledge, and can be integrated to see how systemic parameters influence the entire system. The thermal inputs organized in the hypothalamus to activate thermoregulation responses to heat and cold stimuli, with the widely accepted set-point hypothesis for the regulation of body temperature from a control systems point of view, are, however, not entirely known. There are circumstances (e.g. fever) in which the presumed set-point mechanism appears to break down. This paper evaluates a novel set-level adaptive optimal thermal control paradigm inspired by Hebbian covariance synaptic adaptation, previously proposed based on its potential to predict the homeostatic respiratory system. It introduces a Hebbian feedback covariance learning (HFCL) concept in order to align a neuronal network into the analysis of the thermoregulation system. Hebbian theory is concerned with how neurons connect among themselves to become engrams. The passive-active mathematical model for simulating human thermoregulation during exercise was compared in cool, warm, and hot environments, and then was translated into MATLAB to predict thermoregulation. The two-node core and shell model predictions are comparable with observed thermoregulation responses from the existing literature. The thermoregulation changes with respect to proportionality constant and sensitivity of the receptors. A reasonably general agreement with the measured mean group data of earlier performed laboratory exercise studies was obtained for peak temperature, although it tended to overpredict the core body temperature.

Optimal interaction of respiratory and thermal regulation at rest and during exercise: role of a serotonin-gated spinoparabrachial thermoafferent pathway

Recent evidence indicates that the lateral parabrachial nucleus (LPBN) in dorsolateral pons is pivotal in mediating the feedback control of inspiratory drive by central chemoreceptor input and feedforward control of body temperature by cutaneous thermoreceptor input. The latter is subject to descending serotonergic inhibition which gates the transmission of ascending thermoafferent information from spinal dorsal horn to the LPBN. Here, a model is proposed which suggests that the LPBN may be important in balancing respiratory and thermal homeostasis, two conflicting goals that are heightened by environmental heat/cold stress or exercise where the effects of respiratory thermolysis become prominent. This optimization model of respiratory-thermoregulatory interaction is supported by a host of recent studies which demonstrate that animals with serotonin (5-HT) dysfunction at the spinal dorsal horn--due to 5-HT antagonism, genetic 5-HT defects or spinal cord injury--all display similar respiratory abnormalities that are consistent with hyperactivity of the spinoparabrachial thermoafferent (and pain) pathway.

Homeostasis of exercise hyperpnea and optimal sensorimotor integration: the internal model paradigm

Homeostasis is a basic tenet of biomedicine and an open problem for many physiological control systems. Among them, none has been more extensively studied and intensely debated than the dilemma of exercise hyperpnea - a paradoxical homeostatic increase of respiratory ventilation that is geared to metabolic demands instead of the normal chemoreflex mechanism. Classical control theory has led to a plethora of "feedback/feedforward control" or "set point" hypotheses for homeostatic regulation, yet so far none of them has proved satisfactory in explaining exercise hyperpnea and its interactions with other respiratory inputs. Instead, the available evidence points to a far more sophisticated respiratory controller capable of integrating multiple afferent and efferent signals in adapting the ventilatory pattern toward optimality relative to conflicting homeostatic, energetic and other objectives. This optimality principle parsimoniously mimics exercise hyperpnea, chemoreflex and a host of characteristic respiratory responses to abnormal gas exchange or mechanical loading/unloading in health and in cardiopulmonary diseases - all without resorting to a feedforward "exercise stimulus". Rather, an emergent controller signal encoding the projected metabolic level is predicted by the principle as an exercise-induced 'mental percept' or 'internal model', presumably engendered by associative learning (operant conditioning or classical conditioning) which achieves optimality through continuous identification of, and adaptation to, the causal relationship between respiratory motor output and resultant chemical-mechanical afferent feedbacks. This internal model self-tuning adaptive control paradigm opens a new challenge and exciting opportunity for experimental and theoretical elucidations of the mechanisms of respiratory control - and of homeostatic regulation and sensorimotor integration in general.

Sensorimotor learning and information processing by Bayesian internal models

Fundamental to effective brain-machine interface and neuroprosthesis designs is an understanding of how sensory and motor information are encoded, integrated and adapted by the nervous system. Special session "Neural Information Processing by Bayesian and Internal Models" expounds two current theories of sensorimotor integration which posit that neural information may be encoded centrally as an "internal model" of the environment or as a stochastic state-space model that modulates the activity of spiking neurons. Underlying both theories is a possible role for Bayes' rule--as suggested by the recent findings that the brain may employ Bayesian internal models during certain types of sensorimotor learning in order to optimize task-specific performance and that the emergent activity of certain neural ensembles may be modeled as joint Bayesian point processes. These emerging concepts of neural signal processing have far-reaching implications in applications from rehabilitation engineering to artificial intelligence.

Neural plasticity of respiratory control system: modeling perspectives

Classical models of respiratory control assume a hardwired system architecture with reflex regulation of respiratory rhythm and total ventilation. Recent experimental studies, however, reveal a much more pliable architecture with varying forms of neural plasticity in the afferent and efferent pathways. Here, mathematical models of several types of neural plasticity are proposed and their computational roles in respiratory neural processing are discussed.

A global bioheat model with self-tuning optimal regulation of body temperature using Hebbian feedback covariance learning

In the lower brain, body temperature is continually being regulated almost flawlessly despite huge fluctuations in ambient and physiological conditions that constantly threaten the well-being of the body. The underlying control problem defining thermal homeostasis is one of great enormity: Many systems and sub-systems are involved in temperature regulation and physiological processes are intrinsically complex and intertwined. Thus the defining control system has to take into account the complications of nonlinearities, system uncertainties, delayed feedback loops as well as internal and external disturbances. In this paper, we propose a self-tuning adaptive thermal controller based upon Hebbian feedback covariance learning where the system is to be regulated continually to best suit its environment. This hypothesis is supported in part by postulations of the presence of adaptive optimization behavior in biological systems of certain organisms which face limited resources vital for survival. We demonstrate the use of Hebbian feedback covariance learning as a possible self-adaptive controller in body temperature regulation. The model postulates an important role of Hebbian covariance adaptation as a means of reinforcement learning in the thermal controller. The passive system is based on a simplified 2-node core and shell representation of the body, where global responses are captured. Model predictions are consistent with observed thermoregulatory responses to conditions of exercise and rest, and heat and cold stress. An important implication of the model is that optimal physiological behaviors arising from self-tuning adaptive regulation in the thermal controller may be responsible for the departure from homeostasis in abnormal states, e.g., fever. This was previously unexplained using the conventional "set-point" control theory.

Internal models in sensorimotor integration: perspectives from adaptive control theory

Internal models and adaptive controls are empirical and mathematical paradigms that have evolved separately to describe learning control processes in brain systems and engineering systems, respectively. This paper presents a comprehensive appraisal of the correlation between these paradigms with a view to forging a unified theoretical framework that may benefit both disciplines. It is suggested that the classic equilibrium-point theory of impedance control of arm movement is analogous to continuous gain-scheduling or high-gain adaptive control within or across movement trials, respectively, and that the recently proposed inverse internal model is akin to adaptive sliding control originally for robotic manipulator applications. Modular internal models' architecture for multiple motor tasks is a form of multi-model adaptive control. Stochastic methods, such as generalized predictive control, reinforcement learning, Bayesian learning and Hebbian feedback covariance learning, are reviewed and their possible relevance to motor control is discussed. Possible applicability of a Luenberger observer and an extended Kalman filter to state estimation problems-such as sensorimotor prediction or the resolution of vestibular sensory ambiguity-is also discussed. The important role played by vestibular system identification in postural control suggests an indirect adaptive control scheme whereby system states or parameters are explicitly estimated prior to the implementation of control. This interdisciplinary framework should facilitate the experimental elucidation of the mechanisms of internal models in sensorimotor systems and the reverse engineering of such neural mechanisms into novel brain-inspired adaptive control paradigms in future.

Plasticity of cardiorespiratory neural processing: classification and computational functions

Neural plasticity, or malleability of neuronal structure and function, is an important attribute of the mammalian forebrain and is generally thought to be a kernel of biological intelligence. In this review, we examine some reported manifestations of neural plasticity in the cardiorespiratory system and classify them into four functional categories, integral; differential; memory; and statistical-type plasticity. At the cellular and systems level the myriad forms of cardiorespiratory plasticity display emergent and self-organization properties, use- and disuse-dependent and pairing-specific properties, short-term and long-term potentiation or depression, as well as redundancy in series or parallel structures, convergent pathways or backup and fail-safe surrogate pathways. At the behavioral level, the cardiorespiratory system demonstrates the capability of associative and nonassociative learning, classical and operant conditioning as well as short-term and long-term memory. The remarkable similarity and consistency of the various types of plasticity exhibited at all levels of organization suggest that neural plasticity is integral to cardiorespiratory control and may subserve important physiological functions.