Computational model of the distributed representation of operant reward memory: combinatoric engagement of intrinsic and synaptic plasticity mechanisms

Full Hill-type muscle model of the I1/I3 retractor muscle complex in Aplysia californica

The coordination of complex behavior requires knowledge of both neural dynamics and the mechanics of the periphery. The feeding system of Aplysia californica is an excellent model for investigating questions in soft body systems' neuromechanics because of its experimental tractability. Prior work has attempted to elucidate the mechanical properties of the periphery by using a Hill-type muscle model to characterize the force generation capabilities of the key protractor muscle responsible for moving Aplysia's grasper anteriorly, the I2 muscle. However, the I1/I3 muscle, which is the main driver of retractions of Aplysia's grasper, has not been characterized. Because of the importance of the musculature's properties in generating functional behavior, understanding the properties of muscles like the I1/I3 complex may help to create more realistic simulations of the feeding behavior of Aplysia, which can aid in greater understanding of the neuromechanics of soft-bodied systems. To bridge this gap, in this work, the I1/I3 muscle complex was characterized using force-frequency, length-tension, and force-velocity experiments and showed that a Hill-type model can accurately predict its force-generation properties. Furthermore, the muscle's peak isometric force and stiffness were found to exceed those of the I2 muscle, and these results were analyzed in the context of prior studies on the I1/I3 complex's kinematics in vivo.

A computational neural model that incorporates both intrinsic dynamics and sensory feedback in the Aplysia feeding network

Studying the nervous system underlying animal motor control can shed light on how animals can adapt flexibly to a changing environment. We focus on the neural basis of feeding control in Aplysia californica. Using the Synthetic Nervous System framework, we developed a model of Aplysia feeding neural circuitry that balances neurophysiological plausibility and computational complexity. The circuitry includes neurons, synapses, and feedback pathways identified in existing literature. We organized the neurons into three layers and five subnetworks according to their functional roles. Simulation results demonstrate that the circuitry model can capture the intrinsic dynamics at neuronal and network levels. When combined with a simplified peripheral biomechanical model, it is sufficient to mediate three animal-like feeding behaviors (biting, swallowing, and rejection). The kinematic, dynamic, and neural responses of the model also share similar features with animal data. These results emphasize the functional roles of sensory feedback during feeding.

Persistent modulatory actions and task switching in the feeding network of Aplysia

The activity of multifunctional networks is configured by neuromodulators that exert persistent effects. This raises a question, does this impact the ability of a network to switch from one type of activity to another? We review studies that have addressed this question in the Aplysia feeding circuit. Task switching in this system occurs "asymmetrically." When there is a switch from egestion to ingestion neuromodulation impedes switching (creates a "negative bias"). When there is a switch from ingestion to egestion the biasing is "positive." Ingestion promotes subsequent egestion. We contrast mechanisms responsible for the two types of biasing and show that the observed asymmetry is a consequence of the fact that there is more than one set of egestive circuit parameters.

Sodium-mediated plateau potentials in an identified decisional neuron contribute to feeding-related motor pattern genesis in Aplysia

Motivated behaviors such as feeding depend on the functional properties of decision neurons to provide the flexibility required for behavioral adaptation. Here, we analyzed the ionic basis of the endogenous membrane properties of an identified decision neuron (B63) that drive radula biting cycles underlying food-seeking behavior in Aplysia. Each spontaneous bite cycle arises from the irregular triggering of a plateau-like potential and resultant bursting by rhythmic subthreshold oscillations in B63's membrane potential. In isolated buccal ganglion preparations, and after synaptic isolation, the expression of B63's plateau potentials persisted after removal of extracellular calcium, but was completely suppressed in a tetrodotoxin (TTX)- containing bath solution, thereby indicating the contribution of a transmembrane Na⁺ influx. Potassium outward efflux through tetraethylammonium (TEA)- and calcium-sensitive channels was found to contribute to each plateau's active termination. This intrinsic plateauing capability, in contrast to B63's membrane potential oscillation, was blocked by the calcium-activated non-specific cationic current (I_CAN) blocker flufenamic acid (FFA). Conversely, the SERCA blocker cyclopianozic acid (CPA), which abolished the neuron's oscillation, did not prevent the expression of experimentally evoked plateau potentials. These results therefore indicate that the dynamic properties of the decision neuron B63 rely on two distinct mechanisms involving different sub-populations of ionic conductances.

Variational and phase response analysis for limit cycles with hard boundaries, with applications to neuromechanical control problems

Motor systems show an overall robustness, but because they are highly nonlinear, understanding how they achieve robustness is difficult. In many rhythmic systems, robustness against perturbations involves response of both the shape and the timing of the trajectory. This makes the study of robustness even more challenging. To understand how a motor system produces robust behaviors in a variable environment, we consider a neuromechanical model of motor patterns in the feeding apparatus of the marine mollusk Aplysia californica (Shaw et al. in J Comput Neurosci 38(1):25-51, 2015; Lyttle et al. in Biol Cybern 111(1):25-47, 2017). We established in (Wang et al. in SIAM J Appl Dyn Syst 20(2):701-744, 2021. https://doi.org/10.1137/20M1344974 ) the tools for studying combined shape and timing responses of limit cycle systems under sustained perturbations and here apply them to study robustness of the neuromechanical model against increased mechanical load during swallowing. Interestingly, we discover that nonlinear biomechanical properties confer resilience by immediately increasing resistance to applied loads. In contrast, the effect of changed sensory feedback signal is significantly delayed by the firing rates' hard boundary properties. Our analysis suggests that sensory feedback contributes to robustness in swallowing primarily by shifting the timing of neural activation involved in the power stroke of the motor cycle (retraction). This effect enables the system to generate stronger retractor muscle forces to compensate for the increased load, and hence achieve strong robustness. The approaches that we are applying to understanding a neuromechanical model in Aplysia, and the results that we have obtained, are likely to provide insights into the function of other motor systems that encounter changing mechanical loads and hard boundaries, both due to mechanical and neuronal firing properties.

From Biological Synapses to “Intelligent” Robots

This selective review explores biologically inspired learning as a model for intelligent robot control and sensing technology on the basis of specific examples. Hebbian synaptic learning is discussed as a functionally relevant model for machine learning and intelligence, as explained on the basis of examples from the highly plastic biological neural networks of invertebrates and vertebrates. Its potential for adaptive learning and control without supervision, the generation of functional complexity, and control architectures based on self-organization is brought forward. Learning without prior knowledge based on excitatory and inhibitory neural mechanisms accounts for the process through which survival-relevant or task-relevant representations are either reinforced or suppressed. The basic mechanisms of unsupervised biological learning drive synaptic plasticity and adaptation for behavioral success in living brains with different levels of complexity. The insights collected here point toward the Hebbian model as a choice solution for “intelligent” robotics and sensor systems.

Specific Plasticity Loci and Their Synergism Mediate Operant Conditioning

Despite numerous studies examining the mechanisms of operant conditioning (OC), the diversity of OC plasticity loci and their synergism have not been examined sufficiently. In the well-characterized feeding neural circuit of Aplysia, in vivo and in vitro appetitive OC increases neuronal excitability and electrical coupling among several neurons leading to an increase in expression of ingestive behavior. Here, we used the in vitro analog of OC to investigate whether OC reduces the excitability of a neuron, B4, whose inhibitory connections decrease expression of ingestive behavior. We found OC decreased the excitability of B4. This change appeared intrinsic to B4 because it could be replicated with an analog of OC in isolated cultures of B4 neurons. In addition to changes in B4 excitability, OC decreased the strength of B4's inhibitory connection to a key decision-making neuron, B51. The OC-induced changes were specific without affecting the excitability of another neuron critical for feeding behavior, B8, or the B4-to-B8 inhibitory connection. A conductance-based circuit model indicated that reducing the B4-to-B51 synapse, or increasing B51 excitability, mediated the OC phenotype more effectively than did decreasing B4 excitability. We combined these modifications to examine whether they could act synergistically. Combinations including B51 synergistically enhanced feeding. Taken together, these results suggest modifications of diverse loci work synergistically to mediate OC and that some neurons are well suited to work synergistically with plasticity in other loci.SIGNIFICANCE STATEMENT The ways in which synergism of diverse plasticity loci mediate the change in motor patterns in operant conditioning (OC) are poorly understood. Here, we found that OC was in part mediated by decreasing the intrinsic excitability of a critical neuron of Aplysia feeding behavior, and specifically reducing the strength of one of its inhibitory connections that targets a key decision-making neuron. A conductance-based computational model indicated that the known plasticity loci showed a surprising level of synergism to mediate the behavioral changes associated with OC. These results highlight the importance of understanding the diversity, specificity and synergy among different types of plasticity that encode memory. Also, because OC in Aplysia is mediated by dopamine (DA), the present study provides insights into specific and synergistic mechanisms of DA-mediated reinforcement of behaviors.

Organelle calcium-derived voltage oscillations in pacemaker neurons drive the motor program for food-seeking behavior in Aplysia

The expression of motivated behaviors depends on both external and internally arising neural stimuli, yet the intrinsic releasing mechanisms for such variably occurring behaviors remain elusive. In isolated nervous system preparations of Aplysia, we have found that irregularly expressed cycles of motor output underlying food-seeking behavior arise from regular membrane potential oscillations of varying magnitude in an identified pair of interneurons (B63) in the bilateral buccal ganglia. This rhythmic signal, which is specific to the B63 cells, is generated by organelle-derived intracellular calcium fluxes that activate voltage-independent plasma membrane channels. The resulting voltage oscillation spreads throughout a subset of gap junction-coupled buccal network neurons and by triggering plateau potential-mediated bursts in B63, can initiate motor output driving food-seeking action. Thus, an atypical neuronal pacemaker mechanism, based on rhythmic intracellular calcium store release and intercellular propagation, can act as an autonomous intrinsic releaser for the occurrence of a motivated behavior.

Inferring functional connectivity through graphical directed information

Objective.

Accurate inference of functional connectivity is critical for understanding brain function. Previous methods have limited ability distinguishing between direct and indirect connections because of inadequate scaling with dimensionality. This poor scaling performance reduces the number of nodes that can be included in conditioning. Our goal was to provide a technique that scales better and thereby enables minimization of indirect connections.

Approach.

Our major contribution is a powerful model-free framework, graphical directed information (GDI), that enables pairwise directed functional connections to be conditioned on the activity of substantially more nodes in a network, producing a more accurate graph of functional connectivity that reduces indirect connections. The key technology enabling this advancement is a recent advance in the estimation of mutual information (MI), which relies on multilayer perceptrons and exploiting an alternative representation of the Kullback–Leibler divergence definition of MI. Our second major contribution is the application of this technique to both discretely valued and continuously valued time series.

Main results.

GDI correctly inferred the circuitry of arbitrary Gaussian, nonlinear, and conductance-based networks. Furthermore, GDI inferred many of the connections of a model of a central pattern generator circuit in Aplysia, while also reducing many indirect connections.

Significance.

GDI is a general and model-free technique that can be used on a variety of scales and data types to provide accurate direct connectivity graphs and addresses the critical issue of indirect connections in neural data analysis.

An Anticipatory Circuit Modification That Modifies Subsequent Task Switching

Modulators are generally expected to establish a network configuration that is appropriate for the current circumstances. We characterize a situation where the opposite is apparently observed. A network effect of a peptide modulator is counterproductive in that it tends to impede rather than promote the creation of the configuration that is appropriate when the modulator is released. This raises a question: why does release occur? We present data that strongly suggest that it impacts task switching. Our experiments were conducted in an Aplysia feeding network that generates egestive and ingestive motor programs. Initial experiments focused on egestive activity and the neuron B8. As activity becomes egestive, there is an increase in synaptic drive to B8 and its firing frequency increases (Wang et al., 2019). We show that, as this occurs, there is also a persistent current that develops in B8 that is outward rather than inward. Dynamic clamp introduction of this current decreases excitability. When there is an egestive-ingestive task switch in Aplysia, negative biasing is observed (i.e., a bout of egestive activity has a negative impact on a subsequent attempt to initiate an ingestive response) (Proekt et al., 2004). Using an in vitro analog of negative biasing, we demonstrate that the outward current that develops during egestive priming plays an important role in establishing this phenomenon. Our data suggest that, although the outward current induced as activity becomes egestive is counterproductive at the time, it plays an anticipatory role in that it subsequently impacts task switching.SIGNIFICANCE STATEMENT In this study, we identify a peptide-induced circuit modification (induction of an outward current) that does not immediately promote the establishment of a behaviorally appropriate network configuration. We ask why this might occur, and present data that strongly suggest that it plays an important role during task switching. Specifically, our data suggest that the outward current we characterize plays a role in the negative biasing that is seen in the mollusc Aplysia when there is a transition from egestive to ingestive activity. It is possible that the mechanism that we describe operates in other species. A negative effect of egestion on subsequent ingestion is observed throughout the animal kingdom.

Control for multifunctionality: bioinspired control based on feeding in Aplysia californica

Animals exhibit remarkable feats of behavioral flexibility and multifunctional control that remain challenging for robotic systems. The neural and morphological basis of multifunctionality in animals can provide a source of bioinspiration for robotic controllers. However, many existing approaches to modeling biological neural networks rely on computationally expensive models and tend to focus solely on the nervous system, often neglecting the biomechanics of the periphery. As a consequence, while these models are excellent tools for neuroscience, they fail to predict functional behavior in real time, which is a critical capability for robotic control. To meet the need for real-time multifunctional control, we have developed a hybrid Boolean model framework capable of modeling neural bursting activity and simple biomechanics at speeds faster than real time. Using this approach, we present a multifunctional model of Aplysia californica feeding that qualitatively reproduces three key feeding behaviors (biting, swallowing, and rejection), demonstrates behavioral switching in response to external sensory cues, and incorporates both known neural connectivity and a simple bioinspired mechanical model of the feeding apparatus. We demonstrate that the model can be used for formulating testable hypotheses and discuss the implications of this approach for robotic control and neuroscience.