Generating variable birdsong syllable sequences with branching chain networks in avian premotor nucleus HVC

Lesions in a songbird vocal circuit increase variability in song syntax

Complex skills like speech and dance are composed of ordered sequences of simpler elements, but the neuronal basis for the syntactic ordering of actions is poorly understood. Birdsong is a learned vocal behavior composed of syntactically ordered syllables, controlled in part by the songbird premotor nucleus HVC (proper name). Here, we test whether one of HVC's recurrent inputs, mMAN (medial magnocellular nucleus of the anterior nidopallium), contributes to sequencing in adult male Bengalese finches (Lonchura striata domestica). Bengalese finch song includes several patterns: (1) chunks, comprising stereotyped syllable sequences; (2) branch points, where a given syllable can be followed probabilistically by multiple syllables; and (3) repeat phrases, where individual syllables are repeated variable numbers of times. We found that following bilateral lesions of mMAN, acoustic structure of syllables remained largely intact, but sequencing became more variable, as evidenced by 'breaks' in previously stereotyped chunks, increased uncertainty at branch points, and increased variability in repeat numbers. Our results show that mMAN contributes to the variable sequencing of vocal elements in Bengalese finch song and demonstrate the influence of recurrent projections to HVC. Furthermore, they highlight the utility of species with complex syntax in investigating neuronal control of ordered sequences.

Songbirds can learn flexible contextual control over syllable sequencing

The flexible control of sequential behavior is a fundamental aspect of speech, enabling endless reordering of a limited set of learned vocal elements (syllables or words). Songbirds are phylogenetically distant from humans but share both the capacity for vocal learning and neural circuitry for vocal control that includes direct pallial-brainstem projections. Based on these similarities, we hypothesized that songbirds might likewise be able to learn flexible, moment-by-moment control over vocalizations. Here, we demonstrate that Bengalese finches (Lonchura striata domestica), which sing variable syllable sequences, can learn to rapidly modify the probability of specific sequences (e.g. ‘ab-c’ versus ‘ab-d’) in response to arbitrary visual cues. Moreover, once learned, this modulation of sequencing occurs immediately following changes in contextual cues and persists without external reinforcement. Our findings reveal a capacity in songbirds for learned contextual control over syllable sequencing that parallels human cognitive control over syllable sequencing in speech.

Human speech and birdsong share numerous parallels. Both humans and birds learn their vocalizations during critical phases early in life, and both learn by imitating adults. Moreover, both humans and songbirds possess specific circuits in the brain that connect the forebrain to midbrain vocal centers.

Humans can flexibly control what they say and how by reordering a fixed set of syllables into endless combinations, an ability critical to human speech and language. Birdsongs also vary depending on their context, and melodies to seduce a mate will be different from aggressive songs to warn other males to stay away. However, so far it was unclear whether songbirds are also capable of modifying songs independent of social or other naturally relevant contexts.

To test whether birds can control their songs in a purposeful way, Veit et al. trained adult male Bengalese finches to change the sequence of their songs in response to random colored lights that had no natural meaning to the birds. A specific computer program was used to detect different variations on a theme that the bird naturally produced (for example, “ab-c” versus “ab-d”), and rewarded birds for singing one sequence when the light was yellow, and the other when it was green. Gradually, the finches learned to modify their songs and were able to switch between the appropriate sequences as soon as the light cues changed. This ability persisted for days, even without any further training.

This suggests that songbirds can learn to flexibly and purposefully modify the way in which they sequence the notes in their songs, in a manner that parallels how humans control syllable sequencing in speech. Moreover, birds can learn to do this ‘on command’ in response to an arbitrarily chosen signal, even if it is not something that would impact their song in nature.

Songbirds are an important model to study brain circuits involved in vocal learning. They are one of the few animals that, like humans, learn their vocalizations by imitating conspecifics. The finding that they can also flexibly control vocalizations may help shed light on the interactions between cognitive processing and sophisticated vocal learning abilities.

Addition of new neurons and the emergence of a local neural circuit for precise timing

During development, neurons arrive at local brain areas in an extended period of time, but how they form local neural circuits is unknown. Here we computationally model the emergence of a network for precise timing in the premotor nucleus HVC in songbird. We show that new projection neurons, added to HVC post hatch at early stages of song development, are recruited to the end of a growing feedforward network. High spontaneous activity of the new neurons makes them the prime targets for recruitment in a self-organized process via synaptic plasticity. Once recruited, the new neurons fire readily at precise times, and they become mature. Neurons that are not recruited become silent and replaced by new immature neurons. Our model incorporates realistic HVC features such as interneurons, spatial distributions of neurons, and distributed axonal delays. The model predicts that the birth order of the projection neurons correlates with their burst timing during the song.

Functions of local neural circuits depend on their specific network structures, but how the networks are wired is unknown. We show that such structures can emerge during development through a self-organized process, during which the network is wired by neuron-by-neuron recruitment. This growth is facilitated by steady supply of immature neurons, which are highly excitable and plastic. We suggest that neuron maturation dynamics is an integral part of constructing local neural circuits.

Intrinsic plasticity and birdsong learning

Although information processing and storage in the brain is thought to be primarily orchestrated by synaptic plasticity, other neural mechanisms such as intrinsic plasticity are available. While a number of recent studies have described the plasticity of intrinsic excitability in several types of neurons, the significance of non-synaptic mechanisms in memory and learning remains elusive. After reviewing plasticity of intrinsic excitation in relation to learning and homeostatic mechanisms, we focus on the intrinsic properties of a class of basal-ganglia projecting song system neurons in zebra finch, how these related to each bird's unique learned song, how these properties change over development, and how they are maintained dynamically to rapidly change in response to auditory feedback perturbations. We place these results in the broader theme of learning and changes in intrinsic properties, emphasizing the computational implications of this form of plasticity, which are distinct from synaptic plasticity. The results suggest that exploring reciprocal interactions between intrinsic and network properties will be a fruitful avenue for understanding mechanisms of birdsong learning.

Statistical structure of the trial-to-trial timing variability in synfire chains

Timing and its variability are crucial for behavior. Consequently, neural circuits that take part in the control of timing and in the measurement of temporal intervals have been the subject of much research. Here we provide an analytical and computational account of the temporal variability in what is perhaps the most basic model of a timing circuit-the synfire chain. First we study the statistical structure of trial-to-trial timing variability in a reduced but analytically tractable model: a chain of single integrate-and-fire neurons. We show that this circuit's variability is well described by a generative model consisting of local, global, and jitter components. We relate each of these components to distinct neural mechanisms in the model. Next we establish in simulations that these results carry over to a noisy homogeneous synfire chain. Finally, motivated by the fact that a synfire chain is thought to underlie the circuit that takes part in the control and timing of the zebra finch song, we present simulations of a biologically realistic synfire chain model of the zebra finch timekeeping circuit. We find the structure of trial-to-trial timing variability to be consistent with our previous findings and to agree with experimental observations of the song's temporal variability. Our study therefore provides a possible neuronal account of behavioral variability in zebra finches.

Hidden neural states underlie canary song syntax

Coordinated skills such as speech or dance involve sequences of actions that follow syntactic rules in which transitions between elements depend on the identity and order of past actions. Canary songs are comprised of repeated syllables, called phrases, and the ordering of these phrases follows long-range rules¹, where the choice of what to sing depends on song structure many seconds prior. The neural substrates that support these long-range correlations are unknown. Using miniature head-mounted microscopes and cell-type-specific genetic tools, we observed neural activity in the premotor nucleus HVC^2–4 as canaries explore various phrase sequences in their repertoire. We find neurons that encode past transitions, extending over 4 phrases and spanning up to 4 seconds and 40 syllables. These neurons preferentially encode past actions rather than future actions, can reflect more than a single song history, and occur mostly during the rare phrases that involve history-dependent transitions in song. These findings demonstrate that HVC dynamics includes “hidden states” not reflected in ongoing behavior – states that carry information about prior actions. These states provide a possible substrate to control syntax transitions governed by long-range rules.

Manipulations of inhibition in cortical circuitry differentially affect spectral and temporal features of Bengalese finch song

The interplay between inhibition and excitation can regulate behavioral expression and control, including the expression of communicative behaviors like birdsong. Computational models postulate varying degrees to which inhibition within vocal motor circuitry influences birdsong, but few studies have tested these models by manipulating inhibition. Here we enhanced and attenuated inhibition in the cortical nucleus HVC (used as proper name) of Bengalese finches (Lonchura striata var. domestica). Enhancement of inhibition (with muscimol) in HVC dose-dependently reduced the amount of song produced. Infusions of higher concentrations of muscimol caused some birds to produce spectrally degraded songs, whereas infusions of lower doses of muscimol led to the production of relatively normal (nondegraded) songs. However, the spectral and temporal structures of these nondegraded songs were significantly different from songs produced under control conditions. In particular, muscimol infusions decreased the frequency and amplitude of syllables, increased various measures of acoustic entropy, and increased the variability of syllable structure. Muscimol also increased sequence durations and the variability of syllable timing and syllable sequencing. Attenuation of inhibition (with bicuculline) in HVC led to changes to song distinct from and often opposite to enhancing inhibition. For example, in contrast to muscimol, bicuculline infusions increased syllable amplitude, frequency, and duration and decreased the variability of acoustic features. However, like muscimol, bicuculline increased the variability of syllable sequencing. These data highlight the importance of inhibition to the production of stereotyped vocalizations and demonstrate that changes to neural dynamics within cortical circuitry can differentially affect spectral and temporal features of song.NEW & NOTEWORTHY We reveal that manipulations of inhibition in the cortical nucleus HVC affect the structure, timing, and sequencing of syllables in Bengalese finch song. Enhancing and blocking inhibition led to opposite changes to the acoustic structure and timing of vocalizations, but both caused similar changes to vocal sequencing. These data provide support for computational models of song control but also motivate refinement of existing models to account for differential effects on syllable structure, timing, and sequencing.

Drosophila melanogaster grooming possesses syntax with distinct rules at different temporal scales

Mathematical modeling of behavioral sequences yields insight into the rules and mechanisms underlying sequence generation. Grooming in Drosophila melanogaster is characterized by repeated execution of distinct, stereotyped actions in variable order. Experiments demonstrate that, following stimulation by an irritant, grooming progresses gradually from an early phase dominated by anterior cleaning to a later phase with increased walking and posterior cleaning. We also observe that, at an intermediate temporal scale, there is a strong relationship between the amount of time spent performing body-directed grooming actions and leg-directed actions. We then develop a series of data-driven Markov models that isolate and identify the behavioral features governing transitions between individual grooming bouts. We identify action order as the primary driver of probabilistic, but non-random, syntax structure, as has previously been identified. Subsequent models incorporate grooming bout duration, which also contributes significantly to sequence structure. Our results show that, surprisingly, the syntactic rules underlying probabilistic grooming transitions possess action duration-dependent structure, suggesting that sensory input-independent mechanisms guide grooming behavior at short time scales. Finally, the inclusion of a simple rule that modifies grooming transition probabilities over time yields a generative model that recapitulates the key features of observed grooming sequences at several time scales. These discoveries suggest that sensory input guides action selection by modulating internally generated dynamics. Additionally, the discovery of these principles governing grooming in D. melanogaster demonstrates the utility of incorporating temporal information when characterizing the syntax of behavioral sequences.

Analysis of temporally rich behavioral sequences provides a quantitative description of the rules underlying their generation. Drosophila melanogaster grooming behavior consists of many complex sequences involving repetitions of well-characterized actions. In this paper, we leverage advances in machine vision to automatically annotate over 40 hours of video data of flies covered in dust and develop mathematical models that reveal the existence of syntax in D. melanogaster grooming. We find that sequence organization depends on grooming action identity, as has been well-established, and, more surprisingly, grooming action duration. The discovery of duration-dependent action selection leads us to conclude that, although sensory input informs grooming decisions on long time scales, internal dynamics also guide individual transitions between grooming actions. Therefore, incorporating action duration into our models allows us to uncover multi-scale temporal dynamics that suggest the existence of neural circuits dedicated to partially sensory-independent decision-making. Our approach highlights the importance of incorporating temporal information into sequential models, as doing so reveals the relative contributions of sensory input and internal dynamics to behavioral sequence generation.

Simulation of the song motor pathway in birds: A single neuron initiates a chain of events that produces birdsong with realistic spectra properties

Birdsong is a complex learned behavior regulated by Neuromuscular coordination of different muscle sets necessary for producing relevant sounds. We developed a heterogeneous and stochastically connected neural network representing the pathway from the high vocal center (HVC) to the robust nucleus of the arcopallium (RA) neurons that drive the muscles to generate sounds. We show that a single active neuron is sufficient to initiate a chain of spiking events that results to excite the entire network system. The network could synthesize realistic bird sounds spectra, with spontaneous generation of intermittent sound bursts typical of birdsong (song syllables). This study confirms experiments on animals and on humans, where results have shown that single neurons are responsible for the activation of complex behavior or are associated with high-level perception events.

Morphological characterization of HVC projection neurons in the zebra finch (Taeniopygia guttata)

Singing behavior in the adult male zebra finch is dependent upon the activity of a cortical region known as HVC (proper name). The vast majority of HVC projection neurons send primary axons to either the downstream premotor nucleus RA (robust nucleus of the arcopallium, or primary motor cortex) or Area X (basal ganglia), which play important roles in song production or song learning, respectively. In addition to these long-range outputs, HVC neurons also send local axon collaterals throughout that nucleus. Despite their implications for a range of circuit models, these local processes have never been completely reconstructed. Here, we use in vivo single-neuron Neurobiotin fills to examine 40 projection neurons across 31 birds with somatic positions distributed across HVC. We show that HVC_(RA) and HVC_(X) neurons have categorically distinct dendritic fields. Additionally, these cell classes send axon collaterals that are either restricted to a small portion of HVC ("local neurons") or broadly distributed throughout the entire nucleus ("broadcast neurons"). Overall, these processes within HVC offer a structural basis for significant local processing underlying behaviorally relevant population activity.

Flexibility in motor timing constrains the topology and dynamics of pattern generator circuits

Temporally precise movement patterns underlie many motor skills and innate actions, yet the flexibility with which the timing of such stereotyped behaviors can be modified is poorly understood. To probe this, we induce adaptive changes to the temporal structure of birdsong. We find that the duration of specific song segments can be modified without affecting the timing in other parts of the song. We derive formal prescriptions for how neural networks can implement such flexible motor timing. We find that randomly connected recurrent networks, a common approximation for how neocortex is wired, do not generally conform to these, though certain implementations can approximate them. We show that feedforward networks, by virtue of their one-to-one mapping between network activity and time, are better suited. Our study provides general prescriptions for pattern generator networks that implement flexible motor timing, an important aspect of many motor skills, including birdsong and human speech.

Human speech and bird song requires the generation of precisely timed motor patterns. The authors show that zebra finches can learn to independently modify the duration of individual song segments and find that synfire chain networks are ideally suited to implement such flexible motor timing.

Timing during transitions in Bengalese finch song: implications for motor sequencing

To investigate mechanisms of action sequencing, we examined the relationship between timing and sequencing of syllables in Bengalese finch song. An individual's song comprises acoustically distinct syllables organized into probabilistic sequences: a given syllable potentially can transition to several different syllables (divergence points), and several different syllables can transition to a given syllable (convergence points). In agreement with previous studies, we found that more probable transitions at divergence points occur with shorter intersyllable gaps. One intuition for this relationship is that selection between syllables reflects a competitive branching process, in which stronger links to one syllable lead to both higher probabilities and shorter latencies for transitions to that syllable vs. competing alternatives. However, we found that simulations of competitive race models result in overlapping winning-time distributions for competing outcomes and fail to replicate the strong negative correlation between probability and gap duration found in song data. Further investigation of song structure revealed strong positive correlation between gap durations for transitions that share a common convergent point. Such transitions are not related by a common competitive process, but instead reflect a common terminal syllable. In contrast to gap durations, transition probabilities were not correlated at convergence points. Together, our data suggest that syllable selection happens early during the gap, with gap timing determined chiefly by the latency to syllable initiation. This may result from a process in which probabilistic sequencing is first stabilized, followed by a shortening of the latency to syllables that are sung more often.NEW & NOTEWORTHY Bengalese finch songs consist of probabilistic sequences of syllables. Previous studies revealed a strong negative correlation between transition probability and the duration of intersyllable gaps. We show here that the negative correlation is inconsistent with previous suggestions that timing at syllable transitions is governed by a race between competing alternatives. Rather, the data suggest that syllable selection happens early during the gap, with gap timing determined chiefly by the latency to syllable initiation.

Advantages of comparative studies in songbirds to understand the neural basis of sensorimotor integration

Sensorimotor integration is the process through which the nervous system creates a link between motor commands and associated sensory feedback. This process allows for the acquisition and refinement of many behaviors, including learned communication behaviors such as speech and birdsong. Consequently, it is important to understand fundamental mechanisms of sensorimotor integration, and comparative analyses of this process can provide vital insight. Songbirds offer a powerful comparative model system to study how the nervous system links motor and sensory information for learning and control. This is because the acquisition, maintenance, and control of birdsong critically depend on sensory feedback. Furthermore, there is an incredible diversity of song organizations across songbird species, ranging from songs with simple, stereotyped sequences to songs with complex sequencing of vocal gestures, as well as a wide diversity of song repertoire sizes. Despite this diversity, the neural circuitry for song learning, control, and maintenance remains highly similar across species. Here, we highlight the utility of songbirds for the analysis of sensorimotor integration and the insights about mechanisms of sensorimotor integration gained by comparing different songbird species. Key conclusions from this comparative analysis are that variation in song sequence complexity seems to covary with the strength of feedback signals in sensorimotor circuits and that sensorimotor circuits contain distinct representations of elements in the vocal repertoire, possibly enabling evolutionary variation in repertoire sizes. We conclude our review by highlighting important areas of research that could benefit from increased comparative focus, with particular emphasis on the integration of new technologies.

Temperature Manipulation in Songbird Brain Implicates the Premotor Nucleus HVC in Birdsong Syntax

Variable motor sequences of animals are often structured and can be described by probabilistic transition rules between action elements. Examples include the songs of many songbird species such as the Bengalese finch, which consist of stereotypical syllables sequenced according to probabilistic rules (song syntax). The neural mechanisms behind such rules are poorly understood. Here, we investigate where the song syntax is encoded in the brain of the Bengalese finch by rapidly and reversibly manipulating the temperature in the song production pathway. Cooling the premotor nucleus HVC (proper name) slows down the song tempo, consistent with the idea that HVC controls moment-to-moment timings of acoustic features in the syllables. More importantly, cooling HVC alters the transition probabilities between syllables. Cooling HVC reduces the number of repetitions of long-repeated syllables and increases the randomness of syllable sequences. In contrast, cooling the downstream motor area RA (robust nucleus of the acropallium), which is critical for singing, does not affect the song syntax. Unilateral cooling of HVC shows that control of syllables is mostly lateralized to the left HVC, whereas transition probabilities between the syllables can be affected by cooling HVC in either hemisphere to varying degrees. These results show that HVC is a key site for encoding song syntax in the Bengalese finch. HVC is thus involved both in encoding timings within syllables and in sequencing probabilistic transitions between syllables. Our finding suggests that probabilistic selections and fine-grained timings of action elements can be integrated within the same neural circuits.SIGNIFICANCE STATEMENT Many animal behaviors such as birdsong consist of variable sequences of discrete actions. Where and how the probabilistic rules of such sequences are encoded in the brain is poorly understood. We locally and reversibly cooled brain areas in songbirds during singing. Mild cooling of area HVC in the Bengalese finch brain-a premotor area homologous to the mammalian premotor cortex-alters the statistics of the syllable sequences, suggesting that HVC is critical for birdsong sequences. HVC is also known for controlling moment-to-moment timings within syllables. Our results show that timing and probabilistic sequencing of actions can share the same neural circuits in local brain areas.

Auditory-induced neural dynamics in sensory-motor circuitry predict learned temporal and sequential statistics of birdsong

Predicting future events is a critical computation for both perception and behavior. Despite the essential nature of this computation, there are few studies demonstrating neural activity that predicts specific events in learned, probabilistic sequences. Here, we test the hypotheses that the dynamics of internally generated neural activity are predictive of future events and are structured by the learned temporal-sequential statistics of those events. We recorded neural activity in Bengalese finch sensory-motor area HVC in response to playback of sequences from individuals' songs, and examined the neural activity that continued after stimulus offset. We found that the strength of response to a syllable in the sequence depended on the delay at which that syllable was played, with a maximal response when the delay matched the intersyllable gap normally present for that specific syllable during song production. Furthermore, poststimulus neural activity induced by sequence playback resembled the neural response to the next syllable in the sequence when that syllable was predictable, but not when the next syllable was uncertain. Our results demonstrate that the dynamics of internally generated HVC neural activity are predictive of the learned temporal-sequential structure of produced song and that the strength of this prediction is modulated by uncertainty.

Acoustic sequences in non-human animals: a tutorial review and prospectus

Animal acoustic communication often takes the form of complex sequences, made up of multiple distinct acoustic units. Apart from the well-known example of birdsong, other animals such as insects, amphibians, and mammals (including bats, rodents, primates, and cetaceans) also generate complex acoustic sequences. Occasionally, such as with birdsong, the adaptive role of these sequences seems clear (e.g. mate attraction and territorial defence). More often however, researchers have only begun to characterise - let alone understand - the significance and meaning of acoustic sequences. Hypotheses abound, but there is little agreement as to how sequences should be defined and analysed. Our review aims to outline suitable methods for testing these hypotheses, and to describe the major limitations to our current and near-future knowledge on questions of acoustic sequences. This review and prospectus is the result of a collaborative effort between 43 scientists from the fields of animal behaviour, ecology and evolution, signal processing, machine learning, quantitative linguistics, and information theory, who gathered for a 2013 workshop entitled, 'Analysing vocal sequences in animals'. Our goal is to present not just a review of the state of the art, but to propose a methodological framework that summarises what we suggest are the best practices for research in this field, across taxa and across disciplines. We also provide a tutorial-style introduction to some of the most promising algorithmic approaches for analysing sequences. We divide our review into three sections: identifying the distinct units of an acoustic sequence, describing the different ways that information can be contained within a sequence, and analysing the structure of that sequence. Each of these sections is further subdivided to address the key questions and approaches in that area. We propose a uniform, systematic, and comprehensive approach to studying sequences, with the goal of clarifying research terms used in different fields, and facilitating collaboration and comparative studies. Allowing greater interdisciplinary collaboration will facilitate the investigation of many important questions in the evolution of communication and sociality.

Relationship between the Sequencing and Timing of Vocal Motor Elements in Birdsong

Accurate coordination of the sequencing and timing of motor gestures is important for the performance of complex and evolutionarily relevant behaviors. However, the degree to which motor sequencing and timing are related remains largely unknown. Birdsong is a communicative behavior that consists of discrete vocal motor elements (‘syllables’) that are sequenced and timed in a precise manner. To reveal the relationship between syllable sequencing and timing, we analyzed how variation in the probability of syllable transitions at branch points, nodes in song with variable sequencing across renditions, correlated with variation in the duration of silent gaps between syllable transitions (‘gap durations’) for adult Bengalese finch song. We observed a significant negative relationship between transition probability and gap duration: more prevalent transitions were produced with shorter gap durations. We then assessed the degree to which long-term age-dependent changes and acute context-dependent changes to syllable sequencing and timing followed this inverse relationship. Age- but not context-dependent changes to syllable sequencing and timing were inversely related. On average, gap durations at branch points decreased with age, and the magnitude of this decrease was greater for transitions that increased in prevalence than for transitions that decreased in prevalence. In contrast, there was no systematic relationship between acute context-dependent changes to syllable sequencing and timing. Gap durations at branch points decreased when birds produced female-directed courtship song compared to when they produced undirected song, and the magnitude of this decrease was not related to the direction and magnitude of changes to transition probabilities. These analyses suggest that neural mechanisms that regulate syllable sequencing could similarly control syllable timing but also highlight mechanisms that can independently regulate syllable sequencing and timing.

Dynamic effective connectivity in cortically embedded systems of recurrently coupled synfire chains

As a candidate mechanism of neural representation, large numbers of synfire chains can efficiently be embedded in a balanced recurrent cortical network model. Here we study a model in which multiple synfire chains of variable strength are randomly coupled together to form a recurrent system. The system can be implemented both as a large-scale network of integrate-and-fire neurons and as a reduced model. The latter has binary-state pools as basic units but is otherwise isomorphic to the large-scale model, and provides an efficient tool for studying its behavior. Both the large-scale system and its reduced counterpart are able to sustain ongoing endogenous activity in the form of synfire waves, the proliferation of which is regulated by negative feedback caused by collateral noise. Within this equilibrium, diverse repertoires of ongoing activity are observed, including meta-stability and multiple steady states. These states arise in concert with an effective connectivity structure (ECS). The ECS admits a family of effective connectivity graphs (ECGs), parametrized by the mean global activity level. Of these graphs, the strongly connected components and their associated out-components account to a large extent for the observed steady states of the system. These results imply a notion of dynamic effective connectivity as governing neural computation with synfire chains, and related forms of cortical circuitry with complex topologies.

Neural Sequence Generation Using Spatiotemporal Patterns of Inhibition

Stereotyped sequences of neural activity are thought to underlie reproducible behaviors and cognitive processes ranging from memory recall to arm movement. One of the most prominent theoretical models of neural sequence generation is the synfire chain, in which pulses of synchronized spiking activity propagate robustly along a chain of cells connected by highly redundant feedforward excitation. But recent experimental observations in the avian song production pathway during song generation have shown excitatory activity interacting strongly with the firing patterns of inhibitory neurons, suggesting a process of sequence generation more complex than feedforward excitation. Here we propose a model of sequence generation inspired by these observations in which a pulse travels along a spatially recurrent excitatory chain, passing repeatedly through zones of local feedback inhibition. In this model, synchrony and robust timing are maintained not through redundant excitatory connections, but rather through the interaction between the pulse and the spatiotemporal pattern of inhibition that it creates as it circulates the network. These results suggest that spatially and temporally structured inhibition may play a key role in sequence generation.

An Adapting Auditory-motor Feedback Loop Can Contribute to Generating Vocal Repetition

Consecutive repetition of actions is common in behavioral sequences. Although integration of sensory feedback with internal motor programs is important for sequence generation, if and how feedback contributes to repetitive actions is poorly understood. Here we study how auditory feedback contributes to generating repetitive syllable sequences in songbirds. We propose that auditory signals provide positive feedback to ongoing motor commands, but this influence decays as feedback weakens from response adaptation during syllable repetitions. Computational models show that this mechanism explains repeat distributions observed in Bengalese finch song. We experimentally confirmed two predictions of this mechanism in Bengalese finches: removal of auditory feedback by deafening reduces syllable repetitions; and neural responses to auditory playback of repeated syllable sequences gradually adapt in sensory-motor nucleus HVC. Together, our results implicate a positive auditory-feedback loop with adaptation in generating repetitive vocalizations, and suggest sensory adaptation is important for feedback control of motor sequences.

Repetitions are common in animal vocalizations. Songs of many songbirds contain syllables that repeat a variable number of times, with non-Markovian distributions of repeat counts. The neural mechanism underlying such syllable repetitions is unknown. In this work, we show that auditory feedback plays an important role in sustaining syllable repetitions in the Bengalese finch. Deafening reduces syllable repetitions and skews the repeat number distribution towards short repeats. These effects are explained with our computational model, which suggests that syllable repeats are initially sustained by auditory feedback to the neural networks that drive the syllable production. The feedback strength weakens as the syllable repeats, increasing the likelihood that the syllable repetition stops. Neural recordings confirm such adaptation of auditory feedback to the auditory-motor circuit in the Bengalese finch. Our results suggests that sensory feedback can directly impact repetitions in motor sequences, and may provide insights into neural mechanisms of speech disorders such as stuttering.

Role of the site of synaptic competition and the balance of learning forces for Hebbian encoding of probabilistic Markov sequences

The majority of distinct sensory and motor events occur as temporally ordered sequences with rich probabilistic structure. Sequences can be characterized by the probability of transitioning from the current state to upcoming states (forward probability), as well as the probability of having transitioned to the current state from previous states (backward probability). Despite the prevalence of probabilistic sequencing of both sensory and motor events, the Hebbian mechanisms that mold synapses to reflect the statistics of experienced probabilistic sequences are not well understood. Here, we show through analytic calculations and numerical simulations that Hebbian plasticity (correlation, covariance, and STDP) with pre-synaptic competition can develop synaptic weights equal to the conditional forward transition probabilities present in the input sequence. In contrast, post-synaptic competition can develop synaptic weights proportional to the conditional backward probabilities of the same input sequence. We demonstrate that to stably reflect the conditional probability of a neuron's inputs and outputs, local Hebbian plasticity requires balance between competitive learning forces that promote synaptic differentiation and homogenizing learning forces that promote synaptic stabilization. The balance between these forces dictates a prior over the distribution of learned synaptic weights, strongly influencing both the rate at which structure emerges and the entropy of the final distribution of synaptic weights. Together, these results demonstrate a simple correspondence between the biophysical organization of neurons, the site of synaptic competition, and the temporal flow of information encoded in synaptic weights by Hebbian plasticity while highlighting the utility of balancing learning forces to accurately encode probability distributions, and prior expectations over such probability distributions.

Forebrain circuits underlying the social modulation of vocal communication signals

Across vertebrate species, signalers alter the structure of their communication signals based on the social context. For example, male Bengalese finches produce faster and more stereotyped songs when directing song to females (female-directed [FD] song) than when singing in isolation (undirected [UD] song), and such changes have been found to increase the attractiveness of a male's song. Despite the importance of such social influences, little is known about the mechanisms underlying the social modulation of communication signals. To this end, we analyzed differences in immediate early gene (EGR-1) expression when Bengalese finches produced FD or UD song. Relative to silent birds, EGR-1 expression was elevated in birds producing either FD or UD song throughout vocal control circuitry, including the interface nucleus of the nidopallium (NIf), HVC, the robust nucleus of the arcopallium (RA), Area X, and the lateral magnocellular nucleus of the anterior nidopallium (LMAN). Moreover, EGR-1 expression was higher in HVC, RA, Area X, and LMAN in males producing UD song than in males producing FD song, indicating that social context modulated EGR-1 expression in these areas. However, EGR-1 expression was not significantly different between males producing FD or UD song in NIf, the primary vocal motor input into HVC, suggesting that context-dependent changes could arise de novo in HVC. The pattern of context-dependent differences in EGR-1 expression in the Bengalese finch was highly similar to that in the zebra finch and suggests that social context affects song structure by modulating activity throughout vocal control nuclei.

Juvenile zebra finches learn the underlying structural regularities of their fathers' song

Natural behaviors, such as foraging, tool use, social interaction, birdsong, and language, exhibit branching sequential structure. Such structure should be learnable if it can be inferred from the statistics of early experience. We report that juvenile zebra finches learn such sequential structure in song. Song learning in finches has been extensively studied, and it is generally believed that young males acquire song by imitating tutors (Zann, 1996). Variability in the order of elements in an individual’s mature song occurs, but the degree to which variation in a zebra finch’s song follows statistical regularities has not been quantified, as it has typically been dismissed as production error (Sturdy et al., 1999). Allowing for the possibility that such variation in song is non-random and learnable, we applied a novel analytical approach, based on graph-structured finite-state grammars, to each individual’s full corpus of renditions of songs. This method does not assume syllable-level correspondence between individuals. We find that song variation can be described by probabilistic finite-state graph grammars that are individually distinct, and that the graphs of juveniles are more similar to those of their fathers than to those of other adult males. This grammatical learning is a new parallel between birdsong and language. Our method can be applied across species and contexts to analyze complex variable learned behaviors, as distinct as foraging, tool use, and language.

Independent premotor encoding of the sequence and structure of birdsong in avian cortex

How the brain coordinates rapid sequences of learned behavior, such as human speech, remains a fundamental problem in neuroscience. Birdsong is a model of such behavior, which is learned and controlled by a neural circuit that spans avian cortex, basal ganglia, and thalamus. The songs of adult male zebra finches (Taeniopygia guttata), produced as rapid sequences of vocal gestures (syllables), are encoded by the cortical premotor region HVC (proper name). While the motor encoding of song within HVC has traditionally been viewed as unitary and distributed, we used an ablation technique to ask whether the sequence and structure of song are processed independently within HVC. Results revealed a functional topography across the medial-lateral axis of HVC. Bilateral ablation of medial HVC induced a positive disruption of song (increase in atypical syllable sequences), whereas bilateral ablation of lateral HVC induced a negative disruption (omission of individual syllables). Bilateral ablation of central HVC either had no effect on song or induced syllable omission, similar to lateral HVC ablation. We then investigated HVC connectivity and found parallel afferent and efferent pathways that transit medial and lateral HVC and converge at vocal motor cortex. In light of recent evidence that syntactic and lexical components of human speech are processed independently by neighboring regions of cortex (Menenti et al., 2012), our demonstration of anatomically distinct pathways that differentially process the sequence and structure of birdsong in parallel suggests that the vertebrate brain relies on a common approach to encode rapid sequences of vocal gestures.

Animal vocal sequences: not the Markov chains we thought they were

Many animals produce vocal sequences that appear complex. Most researchers assume that these sequences are well characterized as Markov chains (i.e. that the probability of a particular vocal element can be calculated from the history of only a finite number of preceding elements). However, this assumption has never been explicitly tested. Furthermore, it is unclear how language could evolve in a single step from a Markovian origin, as is frequently assumed, as no intermediate forms have been found between animal communication and human language. Here, we assess whether animal taxa produce vocal sequences that are better described by Markov chains, or by non-Markovian dynamics such as the 'renewal process' (RP), characterized by a strong tendency to repeat elements. We examined vocal sequences of seven taxa: Bengalese finches Lonchura striata domestica, Carolina chickadees Poecile carolinensis, free-tailed bats Tadarida brasiliensis, rock hyraxes Procavia capensis, pilot whales Globicephala macrorhynchus, killer whales Orcinus orca and orangutans Pongo spp. The vocal systems of most of these species are more consistent with a non-Markovian RP than with the Markovian models traditionally assumed. Our data suggest that non-Markovian vocal sequences may be more common than Markov sequences, which must be taken into account when evaluating alternative hypotheses for the evolution of signalling complexity, and perhaps human language origins.

Two neural streams, one voice: pathways for theme and variation in the songbird brain

Birdsong offers a unique model system to understand how a developing brain - once given a set of purely acoustic targets - teaches itself the vocal-tract gestures necessary to imitate those sounds. Like human infants, to juvenile male zebra finches (Taeniopygia guttata) falls the burden of initiating the vocal-motor learning of adult sounds. In both species, adult caregivers provide only a set of sounds to be imitated, with little or no information about the vocal-tract gestures used to produce the sounds. Here, we focus on the central control of birdsong and review the recent discovery that zebra finch song is under dual premotor control. Distinct forebrain pathways for structured (theme) and unstructured (variation) singing not only raise new questions about mechanisms of sensory-motor integration, but also provide a fascinating new research opportunity. A cortical locus for a motor memory of the learned song is now firmly established, meaning that anatomical, physiological, and computational approaches are poised to reveal the neural mechanisms used by the brain to compose the songs of birds.

Vocal motor changes beyond the sensitive period for song plasticity

Behavior is critically shaped during sensitive periods in development. Birdsong is a learned vocal behavior that undergoes dramatic plasticity during a sensitive period of sensorimotor learning. During this period, juvenile songbirds engage in vocal practice to shape their vocalizations into relatively stereotyped songs. By the time songbirds reach adulthood, their songs are relatively stable and thought to be "crystallized." Recent studies, however, highlight the potential for adult song plasticity and suggest that adult song could naturally change over time. As such, we investigated the degree to which temporal and spectral features of song changed over time in adult Bengalese finches. We observed that the sequencing and timing of song syllables became more stereotyped over time. Increases in the stereotypy of syllable sequencing were due to the pruning of infrequently produced transitions and, to a lesser extent, increases in the prevalence of frequently produced transitions. Changes in song tempo were driven by decreases in the duration and variability of intersyllable gaps. In contrast to significant changes to temporal song features, we found little evidence that the spectral structure of adult song syllables changed over time. These data highlight differences in the degree to which temporal and spectral features of adult song change over time and support evidence for distinct mechanisms underlying the control of syllable sequencing, timing, and structure. Furthermore, the observed changes to temporal song features are consistent with a Hebbian framework of behavioral plasticity and support the notion that adult song should be considered a form of vocal practice.

The use of network analysis to study complex animal communication systems: a study on nightingale song

The singing of song birds can form complex signal systems comprised of numerous subunits sung with distinct combinatorial properties that have been described as syntax-like. This complexity has inspired inquiries into similarities of bird song to human language; but the quantitative analysis and description of song sequences is a challenging task. In this study, we analysed song sequences of common nightingales (Luscinia megarhynchos) by means of a network analysis. We translated long nocturnal song sequences into networks of song types with song transitions as connectors. As network measures, we calculated shortest path length and transitivity and identified the 'small-world' character of nightingale song networks. Besides comparing network measures with conventional measures of song complexity, we also found a correlation between network measures and age of birds. Furthermore, we determined the numbers of in-coming and out-going edges of each song type, characterizing transition patterns. These transition patterns were shared across males for certain song types. Playbacks with different transition patterns provided first evidence that these patterns are responded to differently and thus play a role in singing interactions. We discuss potential functions of the network properties of song sequences in the framework of vocal leadership. Network approaches provide biologically meaningful parameters to describe the song structure of species with extremely large repertoires and complex rules of song retrieval.

The respiratory-vocal system of songbirds: anatomy, physiology, and neural control

This wide-ranging review presents an overview of the respiratory-vocal system in songbirds, which are the only other vertebrate group known to display a degree of respiratory control during song rivalling that of humans during speech; this despite the fact that the peripheral components of both the respiratory and vocal systems differ substantially in the two groups. We first provide a brief description of these peripheral components in songbirds (lungs, air sacs and respiratory muscles, vocal organ (syrinx), upper vocal tract) and then proceed to a review of the organization of central respiratory-related neurons in the spinal cord and brainstem, the latter having an organization fundamentally similar to that of the ventral respiratory group of mammals. The second half of the review describes the nature of the motor commands generated in a specialized "cortical" song control circuit and how these might engage brainstem respiratory networks to shape the temporal structure of song. We also discuss a bilaterally projecting "respiratory-thalamic" pathway that links the respiratory system to "cortical" song control nuclei. This necessary pathway for song originates in the brainstem's primary inspiratory center and is hypothesized to play a vital role in synchronizing song motor commands both within and across hemispheres.

Potentiation decay of synapses and length distributions of synfire chains self-organized in recurrent neural networks

Synfire chains are thought to underlie precisely timed sequences of spikes observed in various brain regions and across species. How they are formed is not understood. Here we analyze self-organization of synfire chains through the spike-timing dependent plasticity (STDP) of the synapses, axon remodeling, and potentiation decay of synaptic weights in networks of neurons driven by noisy external inputs and subject to dominant feedback inhibition. Potentiation decay is the gradual, activity-independent reduction of synaptic weights over time. We show that potentiation decay enables a dynamic and statistically stable network connectivity when neurons spike spontaneously. Periodic stimulation of a subset of neurons leads to formation of synfire chains through a random recruitment process, which terminates when the chain connects to itself and forms a loop. We demonstrate that chain length distributions depend on the potentiation decay. Fast potentiation decay leads to long chains with wide distributions, while slow potentiation decay leads to short chains with narrow distributions. We suggest that the potentiation decay, which corresponds to the decay of early long-term potentiation of synapses, is an important synaptic plasticity rule in regulating formation of neural circuity through STDP.

Stepwise acquisition of vocal combinatorial capacity in songbirds and human infants

Human language, as well as birdsong, relies on the ability to arrange vocal elements in novel sequences. However, little is known about the ontogenetic origin of this capacity. We tracked the development of vocal combinatorial capacity in three species of vocal learners, combining an experimental approach in zebra finches with an analysis of natural development of vocal transitions in Bengalese finches and pre-lingual human infants and found a common, stepwise pattern of acquiring vocal transitions across species. In our first study, juvenile zebra finches were trained to perform one song and then the training target was altered, prompting the birds to swap syllable order, or insert a new syllable into a string. All birds solved these permutation tasks in a series of steps, gradually approximating the target sequence by acquiring novel pair-wise syllable transitions, sometimes too slowly to fully accomplish the task. Similarly, in the more complex songs of Bengalese finches, branching points and bidirectional transitions in song-syntax were acquired in a stepwise manner, starting from a more restrictive set of vocal transitions. The babbling of pre-lingual human infants revealed a similar developmental pattern: instead of a single developmental shift from reduplicated to variegated babbling (i.e., from repetitive to diverse sequences), we observed multiple shifts, where each novel syllable type slowly acquired a diversity of pair-wise transitions, asynchronously over development. Collectively, these results point to a common generative process that is conserved across species, suggesting that the long-noted gap between perceptual versus motor combinatorial capabilities in human infants¹ may arise from the challenges in constructing new pair-wise transitions.

Electrophysiological characterization and computational models of HVC neurons in the zebra finch

The nucleus HVC (proper name) within the avian analog of mammal premotor cortex produces stereotyped instructions through the motor pathway leading to precise, learned vocalization by songbirds. Electrophysiological characterization of component HVC neurons is an important requirement in building a model to understand HVC function. The HVC contains three neural populations: neurons that project to the RA (robust nucleus of arcopallium), neurons that project to Area X (of the avian basal ganglia), and interneurons. These three populations are interconnected with specific patterns of excitatory and inhibitory connectivity, and they fire with characteristic patterns both in vivo and in vitro. We performed whole cell current-clamp recordings on HVC neurons within brain slices to examine their intrinsic firing properties and determine which ionic currents are responsible for their characteristic firing patterns. We also developed conductance-based models for the different neurons and calibrated the models using data from our brain slice work. These models were then used to generate predictions about the makeup of the ionic currents that are responsible for the different responses to stimuli. These predictions were then tested and verified in the slice using pharmacological manipulations. The model and the slice work highlight roles of a hyperpolarization-activated inward current (Ih), a low-threshold T-type Ca(2+) current (ICa-T), an A-type K(+) current (IA), a Ca(2+)-activated K(+) current (ISK), and a Na(+)-dependent K(+) current (IKNa) in driving the characteristic neural patterns observed in the three HVC neuronal populations. The result is an improved characterization of the HVC neurons responsible for song production in the songbird.

Long-range order in canary song

Bird songs range in form from the simple notes of a Chipping Sparrow to the rich performance of the nightingale. Non-adjacent correlations can be found in the syntax of some birdsongs, indicating that the choice of what to sing next is determined not only by the current syllable, but also by previous syllables sung. Here we examine the song of the domesticated canary, a complex singer whose song consists of syllables, grouped into phrases that are arranged in flexible sequences. Phrases are defined by a fundamental time-scale that is independent of the underlying syllable duration. We show that the ordering of phrases is governed by long-range rules: the choice of what phrase to sing next in a given context depends on the history of the song, and for some syllables, highly specific rules produce correlations in song over timescales of up to ten seconds. The neural basis of these long-range correlations may provide insight into how complex behaviors are assembled from more elementary, stereotyped modules.

Bird songs range in form from the simple notes of a Chipping Sparrow to the complex repertoire of the nightingale. Recent studies suggest that bird songs may contain non-adjacent dependencies where the choice of what to sing next depends on the history of what has already been produced. However, the complexity of these rules has not been examined statistically for the most elaborate avian singers. Here we show that one complex singer—the domesticated canary—produces a song that is strongly influenced by long-range rules. The choice of how long to repeat a given note or which note to choose next depends on the history of the song, and these dependencies span intervals of time much longer than previously assumed for birdsong. Like most forms of human music, the songs of canaries contain patterns expressed over long timescales, governed by rules that apply to multiple levels of a temporal hierarchy. This vocal complexity provides a valuable model to examine how ordered behaviors are assembled from more elementary neural components in a relatively simple neural circuit.

How language production shapes language form and comprehension

Language production processes can provide insight into how language comprehension works and language typology-why languages tend to have certain characteristics more often than others. Drawing on work in memory retrieval, motor planning, and serial order in action planning, the Production-Distribution-Comprehension (PDC) account links work in the fields of language production, typology, and comprehension: (1) faced with substantial computational burdens of planning and producing utterances, language producers implicitly follow three biases in utterance planning that promote word order choices that reduce these burdens, thereby improving production fluency. (2) These choices, repeated over many utterances and individuals, shape the distributions of utterance forms in language. The claim that language form stems in large degree from producers' attempts to mitigate utterance planning difficulty is contrasted with alternative accounts in which form is driven by language use more broadly, language acquisition processes, or producers' attempts to create language forms that are easily understood by comprehenders. (3) Language perceivers implicitly learn the statistical regularities in their linguistic input, and they use this prior experience to guide comprehension of subsequent language. In particular, they learn to predict the sequential structure of linguistic signals, based on the statistics of previously-encountered input. Thus, key aspects of comprehension behavior are tied to lexico-syntactic statistics in the language, which in turn derive from utterance planning biases promoting production of comparatively easy utterance forms over more difficult ones. This approach contrasts with classic theories in which comprehension behaviors are attributed to innate design features of the language comprehension system and associated working memory. The PDC instead links basic features of comprehension to a different source: production processes that shape language form.

Variable sequencing is actively maintained in a well learned motor skill

Variation in sequencing of actions occurs in many natural behaviors, yet how such variation is maintained is poorly understood. We investigated maintenance of sequence variation in adult Bengalese finch song, a learned skill with rendition-to-rendition variation in the sequencing of discrete syllables (i.e., syllable "b" might transition to "c" with 70% probability and to "d" with 30% probability). We found that probabilities of transitions ordinarily remain stable but could be modified by delivering aversive noise bursts following one transition (e.g., "b→c") but not the alternative (e.g., "b→d"). Such differential reinforcement induced gradual, adaptive decreases in probabilities of targeted transitions and compensatory increases in alternative transitions. Thus, the normal stability of transition probabilities does not reflect hardwired premotor circuitry. While all variable transitions could be modified by differential reinforcement, some were less readily modified than others; these were cases that exhibited more alternation between possible transitions than predicted by chance (i.e., "b→d " would tend to follow "b→c " and vice versa). These history-dependent transitions were less modifiable than more stochastic transitions. Similarly, highly stereotyped transitions (which are completely predictable) were not modifiable. This suggests that stochastically generated variability is crucial for sequence modification. Finally, we found that, when reinforcement ceased, birds gradually restored transition probabilities to their baseline values. Hence, the nervous system retains a representation of baseline probabilities and has the impetus to restore them. Together, our results indicate that variable sequencing in a motor skill can reflect an end point of learning that is stably maintained via continual self-monitoring.

Development of temporal structure in zebra finch song

Zebra finch song has provided an excellent case study in the neural basis of sequence learning, with a high degree of temporal precision and tight links with precisely timed bursting in forebrain neurons. To examine the development of song timing, we measured the following four aspects of song temporal structure at four age ranges between 65 and 375 days posthatch: the mean durations of song syllables and the silent gaps between them, timing variability linked to song tempo, timing variability expressed independently across syllables and gaps, and transition probabilities between consecutive syllable pairs. We found substantial increases in song tempo between 65 and 85 days posthatch, due almost entirely to a shortening of gaps. We also found a decrease in tempo variability, also specific to gaps. Both the magnitude of the increase in tempo and the decrease in tempo variability were correlated on gap-by-gap basis with increases in the reliability of corresponding syllable transitions. Syllables had no systematic increase in tempo or decrease in tempo variability. In contrast to tempo parameters, both syllables and gaps showed an early sharp reduction in independent variability followed by continued reductions over the first year. The data suggest that links between syllable-based representations are strengthened during the later parts of the traditional period of song learning and that song rhythm continues to become more regular throughout the first year of life. Similar learning patterns have been identified in human sequence learning, suggesting a potentially rich area of comparative research.

Triphasic spike-timing-dependent plasticity organizes networks to produce robust sequences of neural activity

Synfire chains have long been proposed to generate precisely timed sequences of neural activity. Such activity has been linked to numerous neural functions including sensory encoding, cognitive and motor responses. In particular, it has been argued that synfire chains underlie the precise spatiotemporal firing patterns that control song production in a variety of songbirds. Previous studies have suggested that the development of synfire chains requires either initial sparse connectivity or strong topological constraints, in addition to any synaptic learning rules. Here, we show that this necessity can be removed by using a previously reported but hitherto unconsidered spike-timing-dependent plasticity (STDP) rule and activity-dependent excitability. Under this rule the network develops stable synfire chains that possess a non-trivial, scalable multi-layer structure, in which relative layer sizes appear to follow a universal function. Using computational modeling and a coarse grained random walk model, we demonstrate the role of the STDP rule in growing, molding and stabilizing the chain, and link model parameters to the resulting structure.

A reafferent and feed-forward model of song syntax generation in the Bengalese finch

Adult Bengalese finches generate a variable song that obeys a distinct and individual syntax. The syntax is gradually lost over a period of days after deafening and is recovered when hearing is restored. We present a spiking neuronal network model of the song syntax generation and its loss, based on the assumption that the syntax is stored in reafferent connections from the auditory to the motor control area. Propagating synfire activity in the HVC codes for individual syllables of the song and priming signals from the auditory network reduce the competition between syllables to allow only those transitions that are permitted by the syntax. Both imprinting of song syntax within HVC and the interaction of the reafferent signal with an efference copy of the motor command are sufficient to explain the gradual loss of syntax in the absence of auditory feedback. The model also reproduces for the first time experimental findings on the influence of altered auditory feedback on the song syntax generation, and predicts song- and species-specific low frequency components in the LFP. This study illustrates how sequential compositionality following a defined syntax can be realized in networks of spiking neurons.

Complex sequencing rules of birdsong can be explained by simple hidden Markov processes

Complex sequencing rules observed in birdsongs provide an opportunity to investigate the neural mechanism for generating complex sequential behaviors. To relate the findings from studying birdsongs to other sequential behaviors such as human speech and musical performance, it is crucial to characterize the statistical properties of the sequencing rules in birdsongs. However, the properties of the sequencing rules in birdsongs have not yet been fully addressed. In this study, we investigate the statistical properties of the complex birdsong of the Bengalese finch (Lonchura striata var. domestica). Based on manual-annotated syllable labeles, we first show that there are significant higher-order context dependencies in Bengalese finch songs, that is, which syllable appears next depends on more than one previous syllable. We then analyze acoustic features of the song and show that higher-order context dependencies can be explained using first-order hidden state transition dynamics with redundant hidden states. This model corresponds to hidden Markov models (HMMs), well known statistical models with a large range of application for time series modeling. The song annotation with these models with first-order hidden state dynamics agreed well with manual annotation, the score was comparable to that of a second-order HMM, and surpassed the zeroth-order model (the Gaussian mixture model; GMM), which does not use context information. Our results imply that the hierarchical representation with hidden state dynamics may underlie the neural implementation for generating complex behavioral sequences with higher-order dependencies.

An imperfect dopaminergic error signal can drive temporal-difference learning

An open problem in the field of computational neuroscience is how to link synaptic plasticity to system-level learning. A promising framework in this context is temporal-difference (TD) learning. Experimental evidence that supports the hypothesis that the mammalian brain performs temporal-difference learning includes the resemblance of the phasic activity of the midbrain dopaminergic neurons to the TD error and the discovery that cortico-striatal synaptic plasticity is modulated by dopamine. However, as the phasic dopaminergic signal does not reproduce all the properties of the theoretical TD error, it is unclear whether it is capable of driving behavior adaptation in complex tasks. Here, we present a spiking temporal-difference learning model based on the actor-critic architecture. The model dynamically generates a dopaminergic signal with realistic firing rates and exploits this signal to modulate the plasticity of synapses as a third factor. The predictions of our proposed plasticity dynamics are in good agreement with experimental results with respect to dopamine, pre- and post-synaptic activity. An analytical mapping from the parameters of our proposed plasticity dynamics to those of the classical discrete-time TD algorithm reveals that the biological constraints of the dopaminergic signal entail a modified TD algorithm with self-adapting learning parameters and an adapting offset. We show that the neuronal network is able to learn a task with sparse positive rewards as fast as the corresponding classical discrete-time TD algorithm. However, the performance of the neuronal network is impaired with respect to the traditional algorithm on a task with both positive and negative rewards and breaks down entirely on a task with purely negative rewards. Our model demonstrates that the asymmetry of a realistic dopaminergic signal enables TD learning when learning is driven by positive rewards but not when driven by negative rewards.

What are the physiological changes that take place in the brain when we solve a problem or learn a new skill? It is commonly assumed that behavior adaptations are realized on the microscopic level by changes in synaptic efficacies. However, this is hard to verify experimentally due to the difficulties of identifying the relevant synapses and monitoring them over long periods during a behavioral task. To address this question computationally, we develop a spiking neuronal network model of actor-critic temporal-difference learning, a variant of reinforcement learning for which neural correlates have already been partially established. The network learns a complex task by means of an internally generated reward signal constrained by recent findings on the dopaminergic system. Our model combines top-down and bottom-up modelling approaches to bridge the gap between synaptic plasticity and system-level learning. It paves the way for further investigations of the dopaminergic system in reward learning in the healthy brain and in pathological conditions such as Parkinson's disease, and can be used as a module in functional models based on brain-scale circuitry.

Cooperation of deterministic dynamics and random noise in production of complex syntactical avian song sequences: a neural network model

How the brain learns and generates temporal sequences is a fundamental issue in neuroscience. The production of birdsongs, a process which involves complex learned sequences, provides researchers with an excellent biological model for this topic. The Bengalese finch in particular learns a highly complex song with syntactical structure. The nucleus HVC (HVC), a premotor nucleus within the avian song system, plays a key role in generating the temporal structures of their songs. From lesion studies, the nucleus interfacialis (NIf) projecting to the HVC is considered one of the essential regions that contribute to the complexity of their songs. However, the types of interaction between the HVC and the NIf that can produce complex syntactical songs remain unclear. In order to investigate the function of interactions between the HVC and NIf, we have proposed a neural network model based on previous biological evidence. The HVC is modeled by a recurrent neural network (RNN) that learns to generate temporal patterns of songs. The NIf is modeled as a mechanism that provides auditory feedback to the HVC and generates random noise that feeds into the HVC. The model showed that complex syntactical songs can be replicated by simple interactions between deterministic dynamics of the RNN and random noise. In the current study, the plausibility of the model is tested by the comparison between the changes in the songs of actual birds induced by pharmacological inhibition of the NIf and the changes in the songs produced by the model resulting from modification of parameters representing NIf functions. The efficacy of the model demonstrates that the changes of songs induced by pharmacological inhibition of the NIf can be interpreted as a trade-off between the effects of noise and the effects of feedback on the dynamics of the RNN of the HVC. These facts suggest that the current model provides a convincing hypothesis for the functional role of NIf-HVC interaction.

A compact statistical model of the song syntax in Bengalese finch

Songs of many songbird species consist of variable sequences of a finite number of syllables. A common approach for characterizing the syntax of these complex syllable sequences is to use transition probabilities between the syllables. This is equivalent to the Markov model, in which each syllable is associated with one state, and the transition probabilities between the states do not depend on the state transition history. Here we analyze the song syntax in Bengalese finch. We show that the Markov model fails to capture the statistical properties of the syllable sequences. Instead, a state transition model that accurately describes the statistics of the syllable sequences includes adaptation of the self-transition probabilities when states are revisited consecutively, and allows associations of more than one state to a given syllable. Such a model does not increase the model complexity significantly. Mathematically, the model is a partially observable Markov model with adaptation (POMMA). The success of the POMMA supports the branching chain network model of how syntax is controlled within the premotor song nucleus HVC, but also suggests that adaptation and many-to-one mapping from the syllable-encoding chain networks in HVC to syllables should be included in the network model.

Hidden Markov models for the stimulus-response relationships of multistate neural systems

Given recent experimental results suggesting that neural circuits may evolve through multiple firing states, we develop a framework for estimating state-dependent neural response properties from spike train data. We modify the traditional hidden Markov model (HMM) framework to incorporate stimulus-driven, non-Poisson point-process observations. For maximal flexibility, we allow external, time-varying stimuli and the neurons' own spike histories to drive both the spiking behavior in each state and the transitioning behavior between states. We employ an appropriately modified expectation-maximization algorithm to estimate the model parameters. The expectation step is solved by the standard forward-backward algorithm for HMMs. The maximization step reduces to a set of separable concave optimization problems if the model is restricted slightly. We first test our algorithm on simulated data and are able to fully recover the parameters used to generate the data and accurately recapitulate the sequence of hidden states. We then apply our algorithm to a recently published data set in which the observed neuronal ensembles displayed multistate behavior and show that inclusion of spike history information significantly improves the fit of the model. Additionally, we show that a simple reformulation of the state space of the underlying Markov chain allows us to implement a hybrid half-multistate, half-histogram model that may be more appropriate for capturing the complexity of certain data sets than either a simple HMM or a simple peristimulus time histogram model alone.

A compositionality machine realized by a hierarchic architecture of synfire chains

The composition of complex behavior is thought to rely on the concurrent and sequential activation of simpler action components, or primitives. Systems of synfire chains have previously been proposed to account for either the simultaneous or the sequential aspects of compositionality; however, the compatibility of the two aspects has so far not been addressed. Moreover, the simultaneous activation of primitives has up until now only been investigated in the context of reactive computations, i.e., the perception of stimuli. In this study we demonstrate how a hierarchical organization of synfire chains is capable of generating both aspects of compositionality for proactive computations such as the generation of complex and ongoing action. To this end, we develop a network model consisting of two layers of synfire chains. Using simple drawing strokes as a visualization of abstract primitives, we map the feed-forward activity of the upper level synfire chains to motion in two-dimensional space. Our model is capable of producing drawing strokes that are combinations of primitive strokes by binding together the corresponding chains. Moreover, when the lower layer of the network is constructed in a closed-loop fashion, drawing strokes are generated sequentially. The generated pattern can be random or deterministic, depending on the connection pattern between the lower level chains. We propose quantitative measures for simultaneity and sequentiality, revealing a wide parameter range in which both aspects are fulfilled. Finally, we investigate the spiking activity of our model to propose candidate signatures of synfire chain computation in measurements of neural activity during action execution.

Neural syntax: cell assemblies, synapsembles, and readers

A widely discussed hypothesis in neuroscience is that transiently active ensembles of neurons, known as "cell assemblies," underlie numerous operations of the brain, from encoding memories to reasoning. However, the mechanisms responsible for the formation and disbanding of cell assemblies and temporal evolution of cell assembly sequences are not well understood. I introduce and review three interconnected topics, which could facilitate progress in defining cell assemblies, identifying their neuronal organization, and revealing causal relationships between assembly organization and behavior. First, I hypothesize that cell assemblies are best understood in light of their output product, as detected by "reader-actuator" mechanisms. Second, I suggest that the hierarchical organization of cell assemblies may be regarded as a neural syntax. Third, constituents of the neural syntax are linked together by dynamically changing constellations of synaptic weights ("synapsembles"). The existing support for this tripartite framework is reviewed and strategies for experimental testing of its predictions are discussed.

Compositionality of arm movements can be realized by propagating synchrony

We present a biologically plausible spiking neuronal network model of free monkey scribbling that reproduces experimental findings on cortical activity and the properties of the scribbling trajectory. The model is based on the idea that synfire chains can encode movement primitives. Here, we map the propagation of activity in a chain to a linearly evolving preferred velocity, which results in parabolic segments that fulfill the two-thirds power law. Connections between chains that match the final velocity of one encoded primitive to the initial velocity of the next allow the composition of random sequences of primitives with smooth transitions. The model provides an explanation for the segmentation of the trajectory and the experimentally observed deviations of the trajectory from the parabolic shape at primitive transition sites. Furthermore, the model predicts low frequency oscillations (<10 Hz) of the motor cortex local field potential during ongoing movements and increasing firing rates of non-specific motor cortex neurons before movement onset.