The basal ganglia is necessary for learning spectral, but not temporal, features of birdsong

Goal-directed vocal planning in a songbird

Songbirds' vocal mastery is impressive, but to what extent is it a result of practice? Can they, based on experienced mismatch with a known target, plan the necessary changes to recover the target in a practice-free manner without intermittently singing? In adult zebra finches, we drive the pitch of a song syllable away from its stable (baseline) variant acquired from a tutor, then we withdraw reinforcement and subsequently deprive them of singing experience by muting or deafening. In this deprived state, birds do not recover their baseline song. However, they revert their songs toward the target by about 1 standard deviation of their recent practice, provided the sensory feedback during the latter signaled a pitch mismatch with the target. Thus, targeted vocal plasticity does not require immediate sensory experience, showing that zebra finches are capable of goal-directed vocal planning.

Lesions in a songbird vocal circuit increase variability in song syntax

Complex skills like speech and dance are composed of ordered sequences of simpler elements, but the neuronal basis for the syntactic ordering of actions is poorly understood. Birdsong is a learned vocal behavior composed of syntactically ordered syllables, controlled in part by the songbird premotor nucleus HVC (proper name). Here, we test whether one of HVC's recurrent inputs, mMAN (medial magnocellular nucleus of the anterior nidopallium), contributes to sequencing in adult male Bengalese finches (Lonchura striata domestica). Bengalese finch song includes several patterns: (1) chunks, comprising stereotyped syllable sequences; (2) branch points, where a given syllable can be followed probabilistically by multiple syllables; and (3) repeat phrases, where individual syllables are repeated variable numbers of times. We found that following bilateral lesions of mMAN, acoustic structure of syllables remained largely intact, but sequencing became more variable, as evidenced by 'breaks' in previously stereotyped chunks, increased uncertainty at branch points, and increased variability in repeat numbers. Our results show that mMAN contributes to the variable sequencing of vocal elements in Bengalese finch song and demonstrate the influence of recurrent projections to HVC. Furthermore, they highlight the utility of species with complex syntax in investigating neuronal control of ordered sequences.

Dopaminergic error signals retune to social feedback during courtship

Hunger, thirst, loneliness and ambition determine the reward value of food, water, social interaction and performance outcome1. Dopamine neurons respond to rewards meeting these diverse needs2-8, but it remains unclear how behaviour and dopamine signals change as priorities change with new opportunities in the environment. One possibility is that dopamine signals for distinct drives are routed to distinct dopamine pathways9,10. Another possibility is that dopamine signals in a given pathway are dynamically tuned to rewards set by the current priority. Here we used electrophysiology and fibre photometry to test how dopamine signals associated with quenching thirst, singing a good song and courting a mate change as male zebra finches (Taeniopygia guttata) were provided with opportunities to retrieve water, evaluate song performance or court a female. When alone, water reward signals were observed in two mesostriatal pathways but singing-related performance error signals were routed to Area X, a striatal nucleus specialized for singing. When courting a female, water seeking was reduced and dopamine responses to both water and song performance outcomes diminished. Instead, dopamine signals in Area X were driven by female calls timed with the courtship song. Thus the dopamine system handled coexisting drives by routing vocal performance and social feedback signals to a striatal area for communication and by flexibly re-tuning to rewards set by the prioritized drive.

Dynamic top-down biasing implements rapid adaptive changes to individual movements

Complex behaviors depend on the coordinated activity of neural ensembles in interconnected brain areas. The behavioral function of such coordination, often measured as co-fluctuations in neural activity across areas, is poorly understood. One hypothesis is that rapidly varying co-fluctuations may be a signature of moment-by-moment task-relevant influences of one area on another. We tested this possibility for error-corrective adaptation of birdsong, a form of motor learning which has been hypothesized to depend on the top-down influence of a higher-order area, LMAN (lateral magnocellular nucleus of the anterior nidopallium), in shaping moment-by-moment output from a primary motor area, RA (robust nucleus of the arcopallium). In paired recordings of LMAN and RA in singing birds, we discovered a neural signature of a top-down influence of LMAN on RA, quantified as an LMAN-leading co-fluctuation in activity between these areas. During learning, this co-fluctuation strengthened in a premotor temporal window linked to the specific movement, sequential context, and acoustic modification associated with learning. Moreover, transient perturbation of LMAN activity specifically within this premotor window caused rapid occlusion of pitch modifications, consistent with LMAN conveying a temporally localized motor-biasing signal. Combined, our results reveal a dynamic top-down influence of LMAN on RA that varies on the rapid timescale of individual movements and is flexibly linked to contexts associated with learning. This finding indicates that inter-area co-fluctuations can be a signature of dynamic top-down influences that support complex behavior and its adaptation.

Learning to pause: Fidelity of and biases in the developmental acquisition of gaps in the communicative signals of a songbird

The temporal organization of sounds used in social contexts can provide information about signal function and evoke varying responses in listeners (receivers). For example, music is a universal and learned human behavior that is characterized by different rhythms and tempos that can evoke disparate responses in listeners. Similarly, birdsong is a social behavior in songbirds that is learned during critical periods in development and used to evoke physiological and behavioral responses in receivers. Recent investigations have begun to reveal the breadth of universal patterns in birdsong and their similarities to common patterns in speech and music, but relatively little is known about the degree to which biological predispositions and developmental experiences interact to shape the temporal patterning of birdsong. Here, we investigated how biological predispositions modulate the acquisition and production of an important temporal feature of birdsong, namely the duration of silent pauses ("gaps") between vocal elements ("syllables"). Through analyses of semi-naturally raised and experimentally tutored zebra finches, we observed that juvenile zebra finches imitate the durations of the silent gaps in their tutor's song. Further, when juveniles were experimentally tutored with stimuli containing a wide range of gap durations, we observed biases in the prevalence and stereotypy of gap durations. Together, these studies demonstrate how biological predispositions and developmental experiences differently affect distinct temporal features of birdsong and highlight similarities in developmental plasticity across birdsong, speech, and music. RESEARCH HIGHLIGHTS: The temporal organization of learned acoustic patterns can be similar across human cultures and across species, suggesting biological predispositions in acquisition. We studied how biological predispositions and developmental experiences affect an important temporal feature of birdsong, namely the duration of silent intervals between vocal elements ("gaps"). Semi-naturally and experimentally tutored zebra finches imitated the durations of gaps in their tutor's song and displayed some biases in the learning and production of gap durations and in gap variability. These findings in the zebra finch provide parallels with the acquisition of temporal features of speech and music in humans.

Motor constellation theory: A model of infants' phonological development

Every normally developing human infant solves the difficult problem of mapping their native-language phonology, but the neural mechanisms underpinning this behavior remain poorly understood. Here, motor constellation theory, an integrative neurophonological model, is presented, with the goal of explicating this issue. It is assumed that infants' motor-auditory phonological mapping takes place through infants' orosensory "reaching" for phonological elements observed in the language-specific ambient phonology, via reference to kinesthetic feedback from motor systems (e.g., articulators), and auditory feedback from resulting speech and speech-like sounds. Attempts are regulated by basal ganglion-cerebellar speech neural circuitry, and successful attempts at reproduction are enforced through dopaminergic signaling. Early in life, the pace of anatomical development constrains mapping such that complete language-specific phonological mapping is prohibited by infants' undeveloped supralaryngeal vocal tract and undescended larynx; constraints gradually dissolve with age, enabling adult phonology. Where appropriate, reference is made to findings from animal and clinical models. Some implications for future modeling and simulation efforts, as well as clinical settings, are also discussed.

Shared mechanisms of auditory and non-auditory vocal learning in the songbird brain

Songbirds and humans share the ability to adaptively modify their vocalizations based on sensory feedback. Prior studies have focused primarily on the role that auditory feedback plays in shaping vocal output throughout life. In contrast, it is unclear how non-auditory information drives vocal plasticity. Here, we first used a reinforcement learning paradigm to establish that somatosensory feedback (cutaneous electrical stimulation) can drive vocal learning in adult songbirds. We then assessed the role of a songbird basal ganglia thalamocortical pathway critical to auditory vocal learning in this novel form of vocal plasticity. We found that both this circuit and its dopaminergic inputs are necessary for non-auditory vocal learning, demonstrating that this pathway is critical for guiding adaptive vocal changes based on both auditory and somatosensory signals. The ability of this circuit to use both auditory and somatosensory information to guide vocal learning may reflect a general principle for the neural systems that support vocal plasticity across species.

Dopamine neurons evaluate natural fluctuations in performance quality

Many motor skills are learned by comparing ongoing behavior to internal performance benchmarks. Dopamine neurons encode performance error in behavioral paradigms where error is externally induced, but it remains unknown whether dopamine also signals the quality of natural performance fluctuations. Here, we record dopamine neurons in singing birds and examine how spontaneous dopamine spiking activity correlates with natural fluctuations in ongoing song. Antidromically identified basal ganglia-projecting dopamine neurons correlate with recent, and not future, song variations, consistent with a role in evaluation, not production. Furthermore, maximal dopamine spiking occurs at a single vocal target, consistent with either actively maintaining the existing song or shifting the song to a nearby form. These data show that spontaneous dopamine spiking can evaluate natural behavioral fluctuations unperturbed by experimental events such as cues or rewards.

Learning and producing skilled behavior requires an internal measure of performance. Duffy et al. examine dopamine neurons’ relationship to natural song in singing birds. Spontaneous dopamine activity correlates with song fluctuations in a manner consistent with evaluation of natural behavioral variations, independent of external perturbations, cues, or rewards.

A Microclip Peripheral Nerve Interface (μcPNI) for Bioelectronic Interfacing with Small Nerves

Peripheral nerves carry sensory (afferent) and motor (efferent) signals between the central nervous system and other parts of the body. The peripheral nervous system (PNS) is therefore rich in targets for therapeutic neuromodulation, bioelectronic medicine, and neuroprosthetics. Peripheral nerve interfaces (PNIs) generally suffer from a tradeoff between selectivity and invasiveness. This work describes the fabrication, evaluation, and chronic implantation in zebra finches of a novel PNI that breaks this tradeoff by interfacing with small nerves. This PNI integrates a soft, stretchable microelectrode array with a 2-photon 3D printed microclip (μcPNI). The advantages of this μcPNI compared to other designs are: a) increased spatial resolution due to bi-layer wiring of the electrode leads, b) reduced mismatch in biomechanical properties with the nerve, c) reduced disturbance to the host tissue due to the small size, d) elimination of sutures or adhesives, e) high circumferential contact with small nerves, f) functionality under considerable strain, and g) graded neuromodulation in a low-threshold stimulation regime. Results demonstrate that the μcPNIs are electromechanically robust, and are capable of reliably recording and stimulating neural activity in vivo in small nerves. The μcPNI may also inform the development of new optical, thermal, ultrasonic, or chemical PNIs as well.

Intrinsic motivation for singing in songbirds is enhanced by temporary singing suppression and regulated by dopamine

Behaviors driven by intrinsic motivation are critical for development and optimization of physical and brain functions, but their underlying mechanisms are not well studied due to the complexity and autonomy of the behavior. Songbirds, such as zebra finches, offer a unique opportunity to study neural substrates of intrinsic motivation because they spontaneously produce many renditions of songs with highly-quantifiable structure for vocal practice, even in the absence of apparent recipients (“undirected singing”). Neural substrates underlying intrinsic motivation for undirected singing are still poorly understood partly because singing motivation cannot be easily manipulated due to its autonomy. Also, undirected singing itself acts as an internal reward, which could increase singing motivation, leading to difficulty in measuring singing motivation independent of singing-associated reward. Here, we report a simple procedure to easily manipulate and quantify intrinsic motivation for undirected singing independent of singing-associated reward. We demonstrate that intrinsic motivation for undirected singing is dramatically enhanced by temporary suppression of singing behavior and the degree of enhancement depends on the duration of suppression. Moreover, by examining latencies to the first song following singing suppression as a measure of singing motivation independent of singing-associated reward, we demonstrate that intrinsic singing motivation is critically regulated by dopamine through D2 receptors. These results provide a simple experimental tool to manipulate and measure the intrinsic motivation for undirected singing and illustrate the importance of zebra finches as a model system to study the neural basis of intrinsically-motivated behaviors.

The basal ganglia control the detailed kinematics of learned motor skills

The basal ganglia are known to influence action selection and modulation of movement vigor, but whether and how they contribute to specifying the kinematics of learned motor skills is not understood. Here, we probe this question by recording and manipulating basal ganglia activity in rats trained to generate complex task-specific movement patterns with rich kinematic structure. We find that the sensorimotor arm of the basal ganglia circuit is crucial for generating the detailed movement patterns underlying the acquired motor skills. Furthermore, the neural representations in the striatum, and the control function they subserve, do not depend on inputs from the motor cortex. Taken together, these results extend our understanding of the basal ganglia by showing that they can specify and control the fine-grained details of learned motor skills through their interactions with lower-level motor circuits.

Unilateral vocal nerve resection alters neurogenesis in the avian song system in a region-specific manner

New neurons born in the adult brain undergo a critical period soon after migration to their site of incorporation. During this time, the behavior of the animal may influence the survival or culling of these cells. In the songbird song system, earlier work suggested that adult-born neurons may be retained in the song motor pathway nucleus HVC with respect to motor progression toward a target song during juvenile song learning, seasonal song restructuring, and experimentally manipulated song variability. However, it is not known whether the quality of song per se, without progressive improvement, may also influence new neuron survival. To test this idea, we experimentally altered song acoustic structure by unilateral denervation of the syrinx, causing a poor quality song. We found no effect of aberrant song on numbers of new neurons in HVC, suggesting that song quality does not influence new neuron culling in this region. However, aberrant song resulted in the loss of left-side dominance in new neurons in the auditory region caudomedial nidopallium (NCM), and a bilateral decrease in new neurons in the basal ganglia nucleus Area X. Thus new neuron culling may be influenced by behavioral feedback in accordance with the function of new neurons within that region. We propose that studying the effects of singing behaviors on new neurons across multiple brain regions that differentially subserve singing may give rise to general rules underlying the regulation of new neuron survival across taxa and brain regions more broadly.

Plasticity of stereotyped birdsong driven by chronic manipulation of cortical-basal ganglia activity

Cortical-basal ganglia (CBG) circuits are critical for motor learning and performance, and are a major site of pathology. In songbirds, a CBG circuit regulates moment-by-moment variability in song and also enables song plasticity. Studies have shown that variable burst firing in LMAN, the output nucleus of this CBG circuit, actively drives acute song variability, but whether and how LMAN drives long-lasting changes in song remains unclear. Here, we ask whether chronic pharmacological augmentation of LMAN bursting is sufficient to drive plasticity in birds singing stereotyped songs. We show that altered LMAN activity drives cumulative changes in acoustic structure, timing, and sequencing over multiple days, and induces repetitions and silent pauses reminiscent of human stuttering. Changes persisted when LMAN was subsequently inactivated, indicating plasticity in song motor regions. Following cessation of pharmacological treatment, acoustic features and song sequence gradually recovered to their baseline values over a period of days to weeks. Together, our findings show that augmented bursting in CBG circuitry drives plasticity in well-learned motor skills, and may inform treatments for basal ganglia movement disorders.

Processes underlying complex patterns of song trait evolution in a Setophaga hybrid zone

During secondary contact between two species when hybrids are less fit than parents, mating signals are expected to diverge, while aggressive signals are expected to converge. If a single signal trait is used in both mating and aggression, then the dynamics between these two forces could influence the evolutionary trajectory of that trait. We studied such a situation in an avian hybrid zone between two Setophaga species, where birdsong is used in both mate attraction and territory defense. We hypothesized that song modules of the two species will show separate and distinct geographic patterns due to the influence of selective pressures for effective territorial aggression and for effective mate attraction. We conducted geographic cline analyses and playback experiments across this hybrid zone. We found an unexpected geographic pattern of asymmetric introgression of song rhythm, which may be explained by results of the playback experiments that suggest that differences in song rhythm serve a greater role in mate attraction than in territory defense. In contrast, differences in syllable morphology show little evidence of importance in mate attraction or territorial defense. Song features converge in the hybrid zone, yet patterns of trait change suggest that the song production modules may vary in their modes of development and inheritance. Syringeal motor gesturing, which gives rise to syllable morphology, shows a nonclinal mosaic pattern, suggesting that this trait may be predominantly learned. In contrast, respiratory patterning, which forms song rhythm, shows a clinal geographic transition, suggesting that this trait could be more innate. The results indicate that opposing forces act independently on song via distinct modules of the song production mechanism, driving complex patterns of song trait evolution.

Expression of FoxP2 in the basal ganglia regulates vocal motor sequences in the adult songbird

Disruption of the transcription factor FoxP2, which is enriched in the basal ganglia, impairs vocal development in humans and songbirds. The basal ganglia are important for the selection and sequencing of motor actions, but the circuit mechanisms governing accurate sequencing of learned vocalizations are unknown. Here, we show that expression of FoxP2 in the basal ganglia is vital for the fluent initiation and termination of birdsong, as well as the maintenance of song syllable sequencing in adulthood. Knockdown of FoxP2 imbalances dopamine receptor expression across striatal direct-like and indirect-like pathways, suggesting a role of dopaminergic signaling in regulating vocal motor sequencing. Confirming this prediction, we show that phasic dopamine activation, and not inhibition, during singing drives repetition of song syllables, thus also impairing fluent initiation and termination of birdsong. These findings demonstrate discrete circuit origins for the dysfluent repetition of vocal elements in songbirds, with implications for speech disorders.

Song learning and plasticity in songbirds

Birdsong provides a fascinating system to study both behavioral and neural plasticity. Oscine songbirds learn to sing, exhibiting behavioral plasticity both during and after the song-learning process. As a bird learns, its song progresses from a plastic and highly variable vocalization into a more stereotyped, crystallized song. However, even after crystallization, song plasticity can occur: some species' songs become more stereotyped over time, whereas other species can incorporate new song elements. Alongside the changes in song, songbirds' brains are also plastic. Both song and neural connections change with the seasons in many species, and new neurons can be added to the song system throughout life. In this review, we highlight important research on behavioral and neural plasticity at multiple timescales, from song development in juveniles to lifelong modifications of learned song.

Actor-critic reinforcement learning in the songbird

It feels rewarding to ace your opponent on match point. Here, we propose common mechanisms underlie reward and performance learning. First, when a singing bird unexpectedly hits the right note, its dopamine (DA) neurons are activated as when a thirsty monkey receives an unexpected juice reward. Second, these DA signals reinforce vocal variations much as they reinforce stimulus-response associations. Third, limbic inputs to DA neurons signal the predicted quality of song syllables much like they signal the predicted reward value of a place or a stimulus during foraging. Finally, songbirds may solve difficult problems in reinforcement learning - such as credit assignment and catastrophic forgetting - with node perturbation and consolidation of reinforced vocal patterns in motor cortical circuits. Consolidation occurs downstream of a canonical 'actor-critic' circuit motif that learns to maximize performance quality in essentially the same way it learns to maximize reward: by computing and learning from prediction errors.

Sensory substitution reveals a manipulation bias

Sensory substitution is a promising therapeutic approach for replacing a missing or diseased sensory organ by translating inaccessible information into another sensory modality. However, many substitution systems are not well accepted by subjects. To explore the effect of sensory substitution on voluntary action repertoires and their associated affective valence, we study deaf songbirds to which we provide visual feedback as a substitute of auditory feedback. Surprisingly, deaf birds respond appetitively to song-contingent binary visual stimuli. They skillfully adapt their songs to increase the rate of visual stimuli, showing that auditory feedback is not required for making targeted changes to vocal repertoires. We find that visually instructed song learning is basal-ganglia dependent. Because hearing birds respond aversively to the same visual stimuli, sensory substitution reveals a preference for actions that elicit sensory feedback over actions that do not, suggesting that substitution systems should be designed to exploit the drive to manipulate.

Statistical structure of the trial-to-trial timing variability in synfire chains

Timing and its variability are crucial for behavior. Consequently, neural circuits that take part in the control of timing and in the measurement of temporal intervals have been the subject of much research. Here we provide an analytical and computational account of the temporal variability in what is perhaps the most basic model of a timing circuit-the synfire chain. First we study the statistical structure of trial-to-trial timing variability in a reduced but analytically tractable model: a chain of single integrate-and-fire neurons. We show that this circuit's variability is well described by a generative model consisting of local, global, and jitter components. We relate each of these components to distinct neural mechanisms in the model. Next we establish in simulations that these results carry over to a noisy homogeneous synfire chain. Finally, motivated by the fact that a synfire chain is thought to underlie the circuit that takes part in the control and timing of the zebra finch song, we present simulations of a biologically realistic synfire chain model of the zebra finch timekeeping circuit. We find the structure of trial-to-trial timing variability to be consistent with our previous findings and to agree with experimental observations of the song's temporal variability. Our study therefore provides a possible neuronal account of behavioral variability in zebra finches.

Social context-dependent singing alters molecular markers of synaptic plasticity signaling in finch basal ganglia Area X

Vocal communication is a crucial skill required throughout life. However, there is a critical gap in our understanding of the underlying molecular brain mechanisms, thereby motivating our use of the zebra finch songbird model. Adult male zebra finches show differences in neural activity patterns in song-dedicated brain nuclei when they sing in two distinct social contexts: a male singing by himself (undirected, UD) and a male singing to a female (female-directed, FD). In our prior work, we showed that in song-dedicated basal ganglia Area X, protein levels of a N-methyl-D-aspartate receptor subtype 2B (NMDAR2B) increased with more UD song and decreased with more FD song. We hypothesized that molecules downstream of this receptor would show differential protein expression levels in Area X between UD and FD song. Specifically, we investigated calcium/calmodulin dependent protein kinase II beta (CaMKIIB), homer scaffold protein 1 (HOMER1), serine/threonine protein kinase (Akt), and mechanistic target of rapamycin kinase (mTOR) following singing and non-singing states in Area X. We show relationships between social context and protein levels. HOMER1 protein levels decreased with time spent singing FD song, and mTOR protein levels decreased with the amount of and time spent singing FD song. For both HOMER1 and mTOR, there were no differences with the amount of UD song. With time spent singing UD, CaMKIIB protein levels trended in a U-shaped curve whereas Akt protein levels trended down. Both molecules showed no change with FD song. Our results support differential involvement of molecules in synaptic plasticity pathways between UD and FD song behaviors.

Printable microscale interfaces for long-term peripheral nerve mapping and precision control

The nascent field of bioelectronic medicine seeks to decode and modulate peripheral nervous system signals to obtain therapeutic control of targeted end organs and effectors. Current approaches rely heavily on electrode-based devices, but size scalability, material and microfabrication challenges, limited surgical accessibility, and the biomechanically dynamic implantation environment are significant impediments to developing and deploying peripheral interfacing technologies. Here, we present a microscale implantable device - the nanoclip - for chronic interfacing with fine peripheral nerves in small animal models that begins to meet these constraints. We demonstrate the capability to make stable, high signal-to-noise ratio recordings of behaviorally-linked nerve activity over multi-week timescales. In addition, we show that multi-channel, current-steering-based stimulation within the confines of the small device can achieve multi-dimensional control of a small nerve. These results highlight the potential of new microscale design and fabrication techniques for realizing viable devices for long-term peripheral interfacing.

Regulation of vocal precision by noradrenergic modulation of a motor nucleus

Norepinephrine (NE) function is often implicated in regulating arousal levels. Recent theory suggests that the noradrenergic system also regulates the optimization of behavior with respect to reward maximization by controlling a switch between exploration and exploitation of the specific actions that yield greatest utility. We show in the songbird that NE can act directly on a cortical motor area and cause a switch between exploratory and exploitative behavior.

Optimal spectral templates for triggered feedback experiments

In the field of songbird neuroscience, researchers have used playback of aversive noise bursts to drive changes in song behavior for specific syllables within a bird’s song. Typically, a short (~5–10 msec) slice of the syllable is selected for targeting and the average spectrum of the slice is used as a template. Sounds that are sufficiently close to the template are considered a match. If other syllables have portions that are spectrally similar to the target, false positive errors will weaken the operant contingency. We present a gradient descent method for template optimization that increases the separation in distance between target and distractors slices, greatly improving targeting accuracy. Applied to songs from five adult Bengalese finches, the fractional reduction in errors for sub-syllabic slices was 51.54±22.92%. At the level of song syllables, we use an error metric that controls for the vastly greater number of distractors vs. target syllables. Setting 5% average error (misses + false positives) as a minimal performance criterion, the number of targetable syllables increased from 3 to 16 out of 61 syllables. At 10% error, targetable syllables increased from 11 to 26. By using simple and robust linear discriminant methods, the algorithm reaches near asymptotic performance when using 10 songs as training data, and the error increases by <2.3% on average when using only a single song for training. Targeting is temporally precise, with average jitter of 3.33 msec for the 16 accurately targeted syllables. Because the algorithm is concerned only with the problem of template selection, it can be used as a simple and robust front end for existing hardware and software implementations for triggered feedback.

Manipulations of inhibition in cortical circuitry differentially affect spectral and temporal features of Bengalese finch song

The interplay between inhibition and excitation can regulate behavioral expression and control, including the expression of communicative behaviors like birdsong. Computational models postulate varying degrees to which inhibition within vocal motor circuitry influences birdsong, but few studies have tested these models by manipulating inhibition. Here we enhanced and attenuated inhibition in the cortical nucleus HVC (used as proper name) of Bengalese finches (Lonchura striata var. domestica). Enhancement of inhibition (with muscimol) in HVC dose-dependently reduced the amount of song produced. Infusions of higher concentrations of muscimol caused some birds to produce spectrally degraded songs, whereas infusions of lower doses of muscimol led to the production of relatively normal (nondegraded) songs. However, the spectral and temporal structures of these nondegraded songs were significantly different from songs produced under control conditions. In particular, muscimol infusions decreased the frequency and amplitude of syllables, increased various measures of acoustic entropy, and increased the variability of syllable structure. Muscimol also increased sequence durations and the variability of syllable timing and syllable sequencing. Attenuation of inhibition (with bicuculline) in HVC led to changes to song distinct from and often opposite to enhancing inhibition. For example, in contrast to muscimol, bicuculline infusions increased syllable amplitude, frequency, and duration and decreased the variability of acoustic features. However, like muscimol, bicuculline increased the variability of syllable sequencing. These data highlight the importance of inhibition to the production of stereotyped vocalizations and demonstrate that changes to neural dynamics within cortical circuitry can differentially affect spectral and temporal features of song.NEW & NOTEWORTHY We reveal that manipulations of inhibition in the cortical nucleus HVC affect the structure, timing, and sequencing of syllables in Bengalese finch song. Enhancing and blocking inhibition led to opposite changes to the acoustic structure and timing of vocalizations, but both caused similar changes to vocal sequencing. These data provide support for computational models of song control but also motivate refinement of existing models to account for differential effects on syllable structure, timing, and sequencing.

Corticobasal ganglia projecting neurons are required for juvenile vocal learning but not for adult vocal plasticity in songbirds

Birdsong, like human speech, consists of a sequence of temporally precise movements acquired through vocal learning. The learning of such sequential vocalizations depends on the neural function of the motor cortex and basal ganglia. However, it is unknown how the connections between cortical and basal ganglia components contribute to vocal motor skill learning, as mammalian motor cortices serve multiple types of motor action and most experimentally tractable animals do not exhibit vocal learning. Here, we leveraged the zebra finch, a songbird, as an animal model to explore the function of the connectivity between cortex-like (HVC) and basal ganglia (area X), connected by HVC_(X) projection neurons with temporally precise firing during singing. By specifically ablating HVC_(X) neurons, juvenile zebra finches failed to copy tutored syllable acoustics and developed temporally unstable songs with less sequence consistency. In contrast, HVC_(X)-ablated adults did not alter their learned song structure, but generated acoustic fluctuations and responded to auditory feedback disruption by the introduction of song deterioration, as did normal adults. These results indicate that the corticobasal ganglia input is important for learning the acoustic and temporal aspects of song structure, but not for generating vocal fluctuations that contribute to the maintenance of an already learned vocal pattern.

Adaptive Regulation of Motor Variability

Trial-to-trial movement variability can both drive motor learning and interfere with expert performance, suggesting benefits of regulating it in context-specific ways. Here we address whether and how the brain regulates motor variability as a function of performance by training rats to execute ballistic forelimb movements for reward. Behavioral datasets comprising millions of trials revealed that motor variability is regulated by two distinct processes. A fast process modulates variability as a function of recent trial outcomes, increasing it when performance is poor and vice versa. A slower process tunes the gain of the fast process based on the uncertainty in the task's reward landscape. Simulations demonstrated that this regulation strategy optimizes reward accumulation over a wide range of time horizons, while also promoting learning. Our results uncover a sophisticated algorithm implemented by the brain to adaptively regulate motor variability to improve task performance. VIDEO ABSTRACT.

Inception of memories that guide vocal learning in the songbird

Animals learn many complex behaviors by emulating the behavior of more experienced individuals. This essential, yet still poorly understood, form of learning relies on the ability to encode lasting memories of observed behaviors. We identified a vocal-motor pathway in the zebra finch where memories that guide learning of song-element durations can be implanted. Activation of synapses in this pathway seeds memories that guide learning of song-element duration and can override learning from social interactions with other individuals. Genetic lesions of this circuit after memory formation, however, do not disrupt subsequent song imitation, which suggests that these memories are stored at downstream synapses. Thus, activity at these sensorimotor synapses can bypass learning from auditory and social experience and embed memories that guide learning of song timing.

Developmental modulation and predictability of age-dependent vocal plasticity in adult zebra finches

Predicting the nature of behavioral plasticity can provide insight into mechanisms of behavioral expression and control. Songbirds like the zebra finch rely on vocal signals for communication, and the performance of these signals demonstrate considerable plasticity over development. Traditionally, these signals were thought to be fixed in adulthood, but recent studies have revealed significant age-dependent changes to spectral and temporal features of song in adult songbirds. A number of age-dependent changes to song resemble acute changes to adult song performance across social contexts (e.g., when an adult male sings to a female relative to when he sings in isolation). The ability of variation in social context-dependent changes to predict variation in age-dependent plasticity would suggest shared mechanisms, but little is known about this predictability. In addition, although developmental experiences can shape adult plasticity, little is known about the extent to which social interactions during development affect age-dependent change to adult song. To this end, we systematically analyzed age- and context-dependent changes to adult zebra finch song, and then examined the degree to which age-dependent changes varied across birds that were social or non-socially tutored birds and to which social context-dependent changes predicted age-dependent changes. Non-socially tutored birds showed more dramatic changes to the broad structure of their motif over time than socially tutored birds, but non-socially and socially tutored birds did not differ in the extent of changes to various spectral and temporal features of song. Overall, we found that adult zebra finches produced longer and more spectrally stereotyped songs when they were older than when they were younger. Moreover, regardless of developmental tutoring, individual variation in age-dependent changes to song bout duration and syllable repetition were predicted by variation in social context-dependent changes to these features. These data indicate that social experiences during development can shape some aspects of adult plasticity and that acute context-dependent and long-term age-dependent changes to some song features could be mediated by modifications within similar neural substrates.

Songbird Ventral Pallidum Sends Diverse Performance Error Signals to Dopaminergic Midbrain

Motor skills improve with practice, requiring outcomes to be evaluated against ever-changing performance benchmarks, yet it remains unclear how performance error signals are computed. Here, we show that the songbird ventral pallidum (VP) is required for song learning and sends diverse song timing and performance error signals to the ventral tegmental area (VTA). Viral tracing revealed inputs to VP from auditory and vocal motor thalamus, auditory and vocal motor cortex, and VTA. Our findings show that VP circuits, commonly associated with hedonic functions, signal performance error during motor sequence learning.

Song Feature Specific Analysis of Isolate Song Reveals Interspecific Variation in Learned Components

Studies of avian vocal development without exposure to conspecific song have been conducted in many passerine species, and the resultant isolate song is often interpreted to represent an expression of the genetic code for conspecific song. There is wide recognition that vocal learning exists in oscine songbirds, but vocal learning has only been thoroughly investigated in a few model species, resulting in a narrow view of birdsong learning. By extracting acoustic signals from published spectrograms, we have reexamined the findings of isolate studies with a universally applicable semi-automated quantitative analysis regimen. When song features were analyzed in light of three different production aspects (respiratory, syringeal, and central programming of sequence), all three show marked interspecific variability in how close isolate song features are to normal. This implies that song learning mechanisms are more variable than is commonly recognized. Our results suggest that the interspecific variation shows no readily observable pattern reflecting phylogeny, which has implications for understanding the mechanisms behind the evolution of avian vocal communication. We emphasize that song learning in passerines provides an excellent opportunity to investigate the evolution of a complex, plastic trait from a phylogenetic perspective.

Variation in sequence dynamics improves maintenance of stereotyped behavior in an example from bird song

Performing a stereotyped behavior successfully over time requires both maintaining performance quality and adapting efficiently to environmental or physical changes affecting performance. The bird song system is a paradigmatic example of learning a stereotyped behavior and therefore is a good place to study the interaction of these two goals. Through a model of bird song learning, we show how instability in neural representation of stable behavior confers advantages for adaptation and maintenance with minimal cost to performance quality. A precise, temporally sparse sequence from the premotor nucleus HVC is crucial to the performance of song in songbirds. We find that learning in the presence of sequence variations facilitates rapid relearning after shifts in the target song or muscle structure and results in decreased error with neuron loss. This robustness is due to the prevention of the buildup of correlations in the learned connectivity. In the absence of sequence variations, these correlations grow, due to the relatively low dimensionality of the exploratory variation in comparison with the number of plastic synapses. Our results suggest one would expect to see variability in neural systems executing stereotyped behaviors, and this variability is an advantageous feature rather than a challenge to overcome.

Neural Competitive Queuing of Ordinal Structure Underlies Skilled Sequential Action

Fluent retrieval and execution of movement sequences is essential for daily activities, but the neural mechanisms underlying sequence planning remain elusive. Here participants learned finger press sequences with different orders and timings and reproduced them in a magneto-encephalography (MEG) scanner. We classified the MEG patterns for each press in the sequence and examined pattern dynamics during preparation and production. Our results demonstrate the “competitive queuing” (CQ) of upcoming action representations, extending previous computational and non-human primate recording studies to non-invasive measures in humans. In addition, we show that CQ reflects an ordinal template that generalizes across specific motor actions at each position. Finally, we demonstrate that CQ predicts participants’ production accuracy and originates from parahippocampal and cerebellar sources. These results suggest that the brain learns and controls multiple sequences by flexibly combining representations of specific actions and interval timing with high-level, parallel representations of sequence position.

•

Non-invasive recordings in humans show “competitive queuing” of upcoming actions

•

Queuing gradient mainly reflects a high-level template of sequence position

•

Queuing gradient originates in ipsilateral parahippocampal and cerebellar areas

•

Strength of queuing gradient predicts production accuracy

Automated home cage training of mice in a hold-still center-out reach task

An obstacle to understanding neural mechanisms of movement is the complex, distributed nature of the mammalian motor system. Here we present a novel behavioral paradigm for high-throughput dissection of neural circuits underlying mouse forelimb control. Custom touch-sensing joysticks were used to quantify mouse forelimb trajectories with micron-millisecond spatiotemporal resolution. Joysticks were integrated into computer-controlled, rack-mountable home cages, enabling batches of mice to be trained in parallel. Closed loop behavioral analysis enabled online control of reward delivery for automated training. We used this system to show that mice can learn, with no human handling, a direction-specific hold-still center-out reach task in which a mouse first held its right forepaw still before reaching out to learned spatial targets. Stabilogram diffusion analysis of submillimeter-scale micromovements produced during the hold demonstrate that an active control process, akin to upright balance, was implemented to maintain forepaw stability. Trajectory decomposition methods, previously used in primates, were used to segment hundreds of thousands of forelimb trajectories into millions of constituent kinematic primitives. This system enables rapid dissection of neural circuits for controlling motion primitives from which forelimb sequences are built. NEW & NOTEWORTHY A novel joystick design resolves mouse forelimb kinematics with micron-millisecond precision. Home cage training is used to train mice in a hold-still center-out reach task. Analytical methods, previously used in primates, are used to decompose mouse forelimb trajectories into kinematic primitives.

The Avian Basal Ganglia Are a Source of Rapid Behavioral Variation That Enables Vocal Motor Exploration

The basal ganglia (BG) participate in aspects of reinforcement learning that require evaluation and selection of motor programs associated with improved performance. However, whether the BG additionally contribute to behavioral variation ("motor exploration") that forms the substrate for such learning remains unclear. In songbirds, a tractable system for studying BG-dependent skill learning, a role for the BG in generating exploratory variability, has been challenged by the finding that lesions of Area X, the song-specific component of the BG, have no lasting effects on several forms of vocal variability that have been studied. Here we demonstrate that lesions of Area X in adult male zebra finches (Taeniopygia gutatta) permanently eliminate rapid within-syllable variation in fundamental frequency (FF), which can act as motor exploration to enable reinforcement-driven song learning. In addition, we found that this within-syllable variation is elevated in juveniles and in adults singing alone, conditions that have been linked to enhanced song plasticity and elevated neural variability in Area X. Consistent with a model that variability is relayed from Area X, via its cortical target, the lateral magnocellular nucleus of the anterior nidopallium (LMAN), to influence song motor circuitry, we found that lesions of LMAN also eliminate within-syllable variability. Moreover, we found that electrical perturbation of LMAN can drive fluctuations in FF that mimic naturally occurring within-syllable variability. Together, these results demonstrate that the BG are a central source of rapid behavioral variation that can serve as motor exploration for vocal learning.SIGNIFICANCE STATEMENT Many complex motor skills, such as speech, are not innately programmed but are learned gradually through trial and error. Learning involves generating exploratory variability in action ("motor exploration") and evaluating subsequent performance to acquire motor programs that lead to improved performance. Although it is well established that the basal ganglia (BG) process signals relating to action evaluation and selection, whether and how the BG promote exploratory motor variability remain unclear. We investigated this question in songbirds, which learn to produce complex vocalizations through trial and error. In contrast with previous studies that did not find effects of BG lesions on vocal motor variability, we demonstrate that the BG are an essential source of rapid behavioral variation linked to vocal learning.

Neuroplasticity in the cerebello-thalamo-basal ganglia pathway: A longitudinal in vivo MRI study in male songbirds

Similar to human speech, bird song is controlled by several pathways including a cortico-basal ganglia-thalamo-cortical (C-BG-T-C) loop. Neurotoxic disengagement of the basal ganglia component, i.e. Area X, induces long-term changes in song performance, while most of the lesioned area regenerates within the first months. Importantly however, the timing and spatial extent of structural neuroplastic events potentially affecting other constituents of the C-BG-T-C loop is not clear. We designed a longitudinal MRI study where changes in brain structure were evaluated relative to the time after neurotoxic lesioning or to vocal performance. By acquiring both Diffusion Tensor Imaging and 3-dimensional anatomical scans, we were able to track alterations in respectively intrinsic tissue properties and local volume. Voxel-based statistical analyses revealed structural remodeling remote to the lesion, i.e. in the thalamus and, surprisingly, the cerebellum, both peaking within the first two months after lesioning Area X. Voxel-wise correlations between song performance and MRI parameters uncovered intriguing brain-behavior relationships in several brain areas pertaining to the C-BG-T-C loop supervising vocal motor control. Our results clearly point to structural neuroplasticity in the cerebellum induced by basal ganglia (striatal) damage and might point to the existence of a human-like cerebello-thalamic-basal ganglia pathway capable of modifying vocal motor output.

Stable Sequential Activity Underlying the Maintenance of a Precisely Executed Skilled Behavior

A vast array of motor skills can be maintained throughout life. Do these behaviors require stability of individual neuron tuning or can the output of a given circuit remain constant despite fluctuations in single cells? This question is difficult to address due to the variability inherent in most motor actions studied in the laboratory. A notable exception, however, is the courtship song of the adult zebra finch, which is a learned, highly precise motor act mediated by orderly dynamics within premotor neurons of the forebrain. By longitudinally tracking the activity of excitatory projection neurons during singing using two-photon calcium imaging, we find that both the number and the precise timing of song-related spiking events remain nearly identical over the span of several weeks to months. These findings demonstrate that learned, complex behaviors can be stabilized by maintaining precise and invariant tuning at the level of single neurons.

Place preference and vocal learning rely on distinct reinforcers in songbirds

In reinforcement learning (RL) agents are typically tasked with maximizing a single objective function such as reward. But it remains poorly understood how agents might pursue distinct objectives at once. In machines, multiobjective RL can be achieved by dividing a single agent into multiple sub-agents, each of which is shaped by agent-specific reinforcement, but it remains unknown if animals adopt this strategy. Here we use songbirds to test if navigation and singing, two behaviors with distinct objectives, can be differentially reinforced. We demonstrate that strobe flashes aversively condition place preference but not song syllables. Brief noise bursts aversively condition song syllables but positively reinforce place preference. Thus distinct behavior-generating systems, or agencies, within a single animal can be shaped by correspondingly distinct reinforcement signals. Our findings suggest that spatially segregated vocal circuits can solve a credit assignment problem associated with multiobjective learning.

A Basal Ganglia Circuit Sufficient to Guide Birdsong Learning

Learning vocal behaviors, like speech and birdsong, is thought to rely on continued performance evaluation. Whether candidate performance evaluation circuits in the brain are sufficient to guide vocal learning is not known. Here, we test the sufficiency of VTA projections to the vocal basal ganglia in singing zebra finches, a songbird species that learns to produce a complex and stereotyped multi-syllabic courtship song during development. We optogenetically manipulate VTA axon terminals in singing birds contingent on how the pitch of an individual song syllable is naturally performed. We find that optical inhibition and excitation of VTA terminals are each sufficient to reliably guide learned changes in song. Inhibition and excitation have opponent effects on future performances of targeted song syllables, consistent with positive and negative reinforcement of performance outcomes. These findings define a central role for reinforcement mechanisms in learning vocalizations and demonstrate minimal circuit elements for learning vocal behaviors. VIDEO ABSTRACT.

Flexibility in motor timing constrains the topology and dynamics of pattern generator circuits

Temporally precise movement patterns underlie many motor skills and innate actions, yet the flexibility with which the timing of such stereotyped behaviors can be modified is poorly understood. To probe this, we induce adaptive changes to the temporal structure of birdsong. We find that the duration of specific song segments can be modified without affecting the timing in other parts of the song. We derive formal prescriptions for how neural networks can implement such flexible motor timing. We find that randomly connected recurrent networks, a common approximation for how neocortex is wired, do not generally conform to these, though certain implementations can approximate them. We show that feedforward networks, by virtue of their one-to-one mapping between network activity and time, are better suited. Our study provides general prescriptions for pattern generator networks that implement flexible motor timing, an important aspect of many motor skills, including birdsong and human speech.

Human speech and bird song requires the generation of precisely timed motor patterns. The authors show that zebra finches can learn to independently modify the duration of individual song segments and find that synfire chain networks are ideally suited to implement such flexible motor timing.

Remodeling of Dendritic Spines in the Avian Vocal Motor Cortex Following Deafening Depends on the Basal Ganglia Circuit

Deafening elicits a deterioration of learned vocalization, in both humans and songbirds. In songbirds, learned vocal plasticity has been shown to depend on the basal ganglia-cortical circuit, but the underlying cellular basis remains to be clarified. Using confocal imaging and electron microscopy, we examined the effect of deafening on dendritic spines in avian vocal motor cortex, the robust nucleus of the arcopallium (RA), and investigated the role of the basal ganglia circuit in motor cortex plasticity. We found rapid structural changes to RA dendritic spines in response to hearing loss, accompanied by learned song degradation. In particular, the morphological characters of RA spine synaptic contacts between 2 major pathways were altered differently. However, experimental disruption of the basal ganglia circuit, through lesions in song-specialized basal ganglia nucleus Area X, largely prevented both the observed changes to RA dendritic spines and the song deterioration after hearing loss. Our results provide cellular evidence to highlight a key role of the basal ganglia circuit in the motor cortical plasticity that underlies learned vocal plasticity.

A common neural circuit mechanism for internally guided and externally reinforced forms of motor learning

The complex skills underlying verbal and musical expression can be learned without external punishment or reward, indicating their learning is internally guided. The neural mechanisms that mediate internally guided learning are poorly understood, but a circuit comprising dopamine-releasing neurons in the midbrain ventral tegmental area (VTA) and their targets in the basal ganglia are important to externally reinforced learning. Juvenile zebra finches copy a tutor song in a process that is internally guided and, in adulthood, can learn to modify the fundamental frequency (pitch) of a target syllable in response to external reinforcement with white noise. Here we combined intersectional genetic ablation of VTA neurons, reversible blockade of dopamine receptors in the basal ganglia, and singing-triggered optogenetic stimulation of VTA terminals to establish that a common VTA-basal ganglia circuit enables internally guided song copying and externally reinforced syllable pitch learning.

Building a state space for song learning

The songbird system has shed light on how the brain produces precisely timed behavioral sequences, and how the brain implements reinforcement learning (RL). RL is a powerful strategy for learning what action to produce in each state, but requires a unique representation of the states involved in the task. Songbird RL circuitry is thought to operate using a representation of each moment within song syllables, consistent with the sparse sequential bursting of neurons in premotor cortical nucleus HVC. However, such sparse sequences are not present in very young birds, which sing highly variable syllables of random lengths. Here, we review and expand upon a model for how the songbird brain could construct latent sequences to support RL, in light of new data elucidating connections between HVC and auditory cortical areas. We hypothesize that learning occurs via four distinct plasticity processes: 1) formation of 'tutor memory' sequences in auditory areas; 2) formation of appropriately-timed latent HVC sequences, seeded by inputs from auditory areas spontaneously replaying the tutor song; 3) strengthening, during spontaneous replay, of connections from HVC to auditory neurons of corresponding timing in the 'tutor memory' sequence, aligning auditory and motor representations for subsequent song evaluation; and 4) strengthening of connections from premotor neurons to motor output neurons that produce the desired sounds, via well-described song RL circuitry.

Songbirds work around computational complexity by learning song vocabulary independently of sequence

While acquiring motor skills, animals transform their plastic motor sequences to match desired targets. However, because both the structure and temporal position of individual gestures are adjustable, the number of possible motor transformations increases exponentially with sequence length. Identifying the optimal transformation towards a given target is therefore a computationally intractable problem. Here we show an evolutionary workaround for reducing the computational complexity of song learning in zebra finches. We prompt juveniles to modify syllable phonology and sequence in a learned song to match a newly introduced target song. Surprisingly, juveniles match each syllable to the most spectrally similar sound in the target, regardless of its temporal position, resulting in unnecessary sequence errors, that they later try to correct. Thus, zebra finches prioritize efficient learning of syllable vocabulary, at the cost of inefficient syntax learning. This strategy provides a non-optimal but computationally manageable solution to the task of vocal sequence learning.

Continuous Time Representations of Song in Zebra Finches

Neurons in the songbird nucleus HVC produce premotor bursts time locked to song with millisecond precision. In this issue of Neuron, Lynch et al. (2016) and Picardo et al. (2016) provide convincing evidence that the population of these bursts contain a continuous representation of time throughout song.