Evidence for a causal inverse model in an avian cortico-basal ganglia circuit

Goal-directed vocal planning in a songbird

Songbirds' vocal mastery is impressive, but to what extent is it a result of practice? Can they, based on experienced mismatch with a known target, plan the necessary changes to recover the target in a practice-free manner without intermittently singing? In adult zebra finches, we drive the pitch of a song syllable away from its stable (baseline) variant acquired from a tutor, then we withdraw reinforcement and subsequently deprive them of singing experience by muting or deafening. In this deprived state, birds do not recover their baseline song. However, they revert their songs toward the target by about 1 standard deviation of their recent practice, provided the sensory feedback during the latter signaled a pitch mismatch with the target. Thus, targeted vocal plasticity does not require immediate sensory experience, showing that zebra finches are capable of goal-directed vocal planning.

Dynamic top-down biasing implements rapid adaptive changes to individual movements

Complex behaviors depend on the coordinated activity of neural ensembles in interconnected brain areas. The behavioral function of such coordination, often measured as co-fluctuations in neural activity across areas, is poorly understood. One hypothesis is that rapidly varying co-fluctuations may be a signature of moment-by-moment task-relevant influences of one area on another. We tested this possibility for error-corrective adaptation of birdsong, a form of motor learning which has been hypothesized to depend on the top-down influence of a higher-order area, LMAN (lateral magnocellular nucleus of the anterior nidopallium), in shaping moment-by-moment output from a primary motor area, RA (robust nucleus of the arcopallium). In paired recordings of LMAN and RA in singing birds, we discovered a neural signature of a top-down influence of LMAN on RA, quantified as an LMAN-leading co-fluctuation in activity between these areas. During learning, this co-fluctuation strengthened in a premotor temporal window linked to the specific movement, sequential context, and acoustic modification associated with learning. Moreover, transient perturbation of LMAN activity specifically within this premotor window caused rapid occlusion of pitch modifications, consistent with LMAN conveying a temporally localized motor-biasing signal. Combined, our results reveal a dynamic top-down influence of LMAN on RA that varies on the rapid timescale of individual movements and is flexibly linked to contexts associated with learning. This finding indicates that inter-area co-fluctuations can be a signature of dynamic top-down influences that support complex behavior and its adaptation.

Wild nightingales flexibly match whistle pitch in real time

Interactive vocal communication, similar to a human conversation, requires flexible and real-time changes to vocal output in relation to preceding auditory stimuli. These vocal adjustments are essential to ensuring both the suitable timing and content of the interaction. Precise timing of dyadic vocal exchanges has been investigated in a variety of species, including humans. In contrast, the ability of non-human animals to accurately adjust specific spectral features of vocalization extemporaneously in response to incoming auditory information is less well studied. One spectral feature of acoustic signals is the fundamental frequency, which we perceive as pitch. Many animal species can discriminate between sound frequencies, but real-time detection and reproduction of an arbitrary pitch have only been observed in humans. Here, we show that nightingales in the wild can match the pitch of whistle songs while singing in response to conspecifics or pitch-controlled whistle playbacks. Nightingales matched whistles across their entire pitch production range indicating that they can flexibly tune their vocal output along a wide continuum. Prompt whistle pitch matches were more precise than delayed ones, suggesting the direct mapping of auditory information onto a motor command to achieve online vocal replication of a heard pitch. Although nightingales' songs follow annual cycles of crystallization and deterioration depending on breeding status, the observed pitch-matching behavior is present year-round, suggesting a stable neural circuit independent of seasonal changes in physiology. Our findings represent the first case of non-human instantaneous vocal imitation of pitch, highlighting a promising model for understanding sensorimotor transformation within an interactive context. VIDEO ABSTRACT.

Self-organization of songbird neural sequences during social isolation

Behaviors emerge via a combination of experience and innate predispositions. As the brain matures, it undergoes major changes in cellular, network, and functional properties that can be due to sensory experience as well as developmental processes. In normal birdsong learning, neural sequences emerge to control song syllables learned from a tutor. Here, we disambiguate the role of tutor experience and development in neural sequence formation by delaying exposure to a tutor. Using functional calcium imaging, we observe neural sequences in the absence of tutoring, demonstrating that tutor experience is not necessary for the formation of sequences. However, after exposure to a tutor, pre-existing sequences can become tightly associated with new song syllables. Since we delayed tutoring, only half our birds learned new syllables following tutor exposure. The birds that failed to learn were the birds in which pre-tutoring neural sequences were most 'crystallized,' that is, already tightly associated with their (untutored) song.

How the conception of control influences our understanding of actions

Wilful movement requires neural control. Commonly, neural computations are thought to generate motor commands that bring the musculoskeletal system - that is, the plant - from its current physical state into a desired physical state. The current state can be estimated from past motor commands and from sensory information. Modelling movement on the basis of this concept of plant control strives to explain behaviour by identifying the computational principles for control signals that can reproduce the observed features of movements. From an alternative perspective, movements emerge in a dynamically coupled agent-environment system from the pursuit of subjective perceptual goals. Modelling movement on the basis of this concept of perceptual control aims to identify the controlled percepts and their coupling rules that can give rise to the observed characteristics of behaviour. In this Perspective, we discuss a broad spectrum of approaches to modelling human motor control and their notions of control signals, internal models, handling of sensory feedback delays and learning. We focus on the influence that the plant control and the perceptual control perspective may have on decisions when modelling empirical data, which may in turn shape our understanding of actions.

Rhythmically bursting songbird vocomotor neurons are organized into multiple sequences, suggesting a network/intrinsic properties model encoding song and error, not time

In zebra finch, basal ganglia projecting "HVC _X " neurons emit one or more spike bursts during each song motif (canonical sequence of syllables), which are thought to be driven in part by a process of spike rebound excitation. Zebra finch songs are highly stereotyped and recent results indicate that the intrinsic properties of HVC _X neurons are similar within each bird, vary among birds depending on similarity of the songs, and vary with song errors. We tested the hypothesis that the timing of spike bursts during singing also evince individual-specific distributions. Examining previously published data, we demonstrated that the intervals between bursts of multibursting HVC _X are similar for neurons within each bird, in many cases highly clustered at distinct peaks, with the patterns varying among birds. The fixed delay between bursts and different times when neurons are first recruited in the song yields precisely timed multiple sequences of bursts throughout the song, not the previously envisioned single sequence of bursts treated as events having statistically independent timing. A given moment in time engages multiple sequences and both single bursting and multibursting HVC _X simultaneously. This suggests a model where a population of HVC _X sharing common intrinsic properties driving spike rebound excitation influence the timing of a given HVC _X burst through lateral inhibitory interactions. Perturbations in burst timing, representing error, could propagate in time. Our results extend the concept of central pattern generators to complex vertebrate vocal learning and suggest that network activity (timing of inhibition) and HVC _X intrinsic properties become coordinated during developmental birdsong learning.

Neural correlates of vocal initiation in the VTA/SNc of juvenile male zebra finches

Initiation and execution of complex learned vocalizations such as human speech and birdsong depend on multiple brain circuits. In songbirds, neurons in the motor cortices and basal ganglia circuitry exhibit preparatory activity before initiation of song, and that activity is thought to play an important role in successful song performance. However, it remains unknown where a start signal for song is represented in the brain and how such a signal would lead to appropriate vocal initiation. To test whether neurons in the midbrain ventral tegmental area (VTA) and substantia nigra pars compacta (SNc) show activity related to song initiation, we carried out extracellular recordings of VTA/SNc single units in singing juvenile male zebra finches. We found that a subset of VTA/SNc units exhibit phasic activity precisely time-locked to the onset of the song bout, and that the activity occurred specifically at the beginning of song. These findings suggest that phasic activity in the VTA/SNc represents a start signal that triggers song vocalization.

The multi-dimensional nature of vocal learning

How learning affects vocalizations is a key question in the study of animal communication and human language. Parallel efforts in birds and humans have taught us much about how vocal learning works on a behavioural and neurobiological level. Subsequent efforts have revealed a variety of cases among mammals in which experience also has a major influence on vocal repertoires. Janik and Slater (Anim. Behav.60, 1-11. (doi:10.1006/anbe.2000.1410)) introduced the distinction between vocal usage and production learning, providing a general framework to categorize how different types of learning influence vocalizations. This idea was built on by Petkov and Jarvis (Front. Evol. Neurosci.4, 12. (doi:10.3389/fnevo.2012.00012)) to emphasize a more continuous distribution between limited and more complex vocal production learners. Yet, with more studies providing empirical data, the limits of the initial frameworks become apparent. We build on these frameworks to refine the categorization of vocal learning in light of advances made since their publication and widespread agreement that vocal learning is not a binary trait. We propose a novel classification system, based on the definitions by Janik and Slater, that deconstructs vocal learning into key dimensions to aid in understanding the mechanisms involved in this complex behaviour. We consider how vocalizations can change without learning, and a usage learning framework that considers context specificity and timing. We identify dimensions of vocal production learning, including the copying of auditory models (convergence/divergence on model sounds, accuracy of copying), the degree of change (type and breadth of learning) and timing (when learning takes place, the length of time it takes and how long it is retained). We consider grey areas of classification and current mechanistic understanding of these behaviours. Our framework identifies research needs and will help to inform neurobiological and evolutionary studies endeavouring to uncover the multi-dimensional nature of vocal learning. This article is part of the theme issue 'Vocal learning in animals and humans'.

Neurally driven synthesis of learned, complex vocalizations

Brain machine interfaces (BMIs) hold promise to restore impaired motor function and serve as powerful tools to study learned motor skill. While limb-based motor prosthetic systems have leveraged nonhuman primates as an important animal model,^1-4 speech prostheses lack a similar animal model and are more limited in terms of neural interface technology, brain coverage, and behavioral study design.^5-7 Songbirds are an attractive model for learned complex vocal behavior. Birdsong shares a number of unique similarities with human speech,^8-10 and its study has yielded general insight into multiple mechanisms and circuits behind learning, execution, and maintenance of vocal motor skill.^11-18 In addition, the biomechanics of song production bear similarity to those of humans and some nonhuman primates.^19-23 Here, we demonstrate a vocal synthesizer for birdsong, realized by mapping neural population activity recorded from electrode arrays implanted in the premotor nucleus HVC onto low-dimensional compressed representations of song, using simple computational methods that are implementable in real time. Using a generative biomechanical model of the vocal organ (syrinx) as the low-dimensional target for these mappings allows for the synthesis of vocalizations that match the bird's own song. These results provide proof of concept that high-dimensional, complex natural behaviors can be directly synthesized from ongoing neural activity. This may inspire similar approaches to prosthetics in other species by exploiting knowledge of the peripheral systems and the temporal structure of their output.

Actor-critic reinforcement learning in the songbird

It feels rewarding to ace your opponent on match point. Here, we propose common mechanisms underlie reward and performance learning. First, when a singing bird unexpectedly hits the right note, its dopamine (DA) neurons are activated as when a thirsty monkey receives an unexpected juice reward. Second, these DA signals reinforce vocal variations much as they reinforce stimulus-response associations. Third, limbic inputs to DA neurons signal the predicted quality of song syllables much like they signal the predicted reward value of a place or a stimulus during foraging. Finally, songbirds may solve difficult problems in reinforcement learning - such as credit assignment and catastrophic forgetting - with node perturbation and consolidation of reinforced vocal patterns in motor cortical circuits. Consolidation occurs downstream of a canonical 'actor-critic' circuit motif that learns to maximize performance quality in essentially the same way it learns to maximize reward: by computing and learning from prediction errors.

An avian cortical circuit for chunking tutor song syllables into simple vocal-motor units

How are brain circuits constructed to achieve complex goals? The brains of young songbirds develop motor circuits that achieve the goal of imitating a specific tutor song to which they are exposed. Here, we set out to examine how song-generating circuits may be influenced early in song learning by a cortical region (NIf) at the interface between auditory and motor systems. Single-unit recordings reveal that, during juvenile babbling, NIf neurons burst at syllable onsets, with some neurons exhibiting selectivity for particular emerging syllable types. When juvenile birds listen to their tutor, NIf neurons are also activated at tutor syllable onsets, and are often selective for particular syllable types. We examine a simple computational model in which tutor exposure imprints the correct number of syllable patterns as ensembles in an interconnected NIf network. These ensembles are then reactivated during singing to train a set of syllable sequences in the motor network.

Young songbirds learn to imitate their parents’ songs. Here, the authors find that, in baby birds, neurons in a brain region at the interface of auditory and motor circuits signal the onsets of song syllables during both tutoring and babbling, suggesting a specific neural mechanism for vocal imitation.

Undirected singing rate as a non-invasive tool for welfare monitoring in isolated male zebra finches

Research on the songbird zebra finch (Taeniopygia guttata) has advanced our behavioral, hormonal, neuronal, and genetic understanding of vocal learning. However, little is known about the impact of typical experimental manipulations on the welfare of these birds. Here we explore whether the undirected singing rate can be used as an indicator of welfare. We tested this idea by performing a post hoc analysis of singing behavior in isolated male zebra finches subjected to interactive white noise, to surgery, or to tethering. We find that the latter two experimental manipulations transiently but reliably decreased singing rates. By contraposition, we infer that a high-sustained singing rate is suggestive of successful coping or improved welfare in these experiments. Our analysis across more than 300 days of song data suggests that a singing rate above a threshold of several hundred song motifs per day implies an absence of an acute stressor or a successful coping with stress. Because singing rate can be measured in a completely automatic fashion, its observation can help to reduce experimenter bias in welfare monitoring. Because singing rate measurements are non-invasive, we expect this study to contribute to the refinement of the current welfare monitoring tools in zebra finches.

Hierarchical motor control in mammals and machines

Advances in artificial intelligence are stimulating interest in neuroscience. However, most attention is given to discrete tasks with simple action spaces, such as board games and classic video games. Less discussed in neuroscience are parallel advances in "synthetic motor control". While motor neuroscience has recently focused on optimization of single, simple movements, AI has progressed to the generation of rich, diverse motor behaviors across multiple tasks, at humanoid scale. It is becoming clear that specific, well-motivated hierarchical design elements repeatedly arise when engineering these flexible control systems. We review these core principles of hierarchical control, relate them to hierarchy in the nervous system, and highlight research themes that we anticipate will be critical in solving challenges at this disciplinary intersection.

Discrete Evaluative and Premotor Circuits Enable Vocal Learning in Songbirds

Virtuosic motor performance requires the ability to evaluate and modify individual gestures within a complex motor sequence. Where and how the evaluative and premotor circuits operate within the brain to enable such temporally precise learning is poorly understood. Songbirds can learn to modify individual syllables within their complex vocal sequences, providing a system for elucidating the underlying evaluative and premotor circuits. We combined behavioral and optogenetic methods to identify 2 afferents to the ventral tegmental area (VTA) that serve evaluative roles in syllable-specific learning and to establish that downstream cortico-basal ganglia circuits serve a learning role that is only premotor. Furthermore, song performance-contingent optogenetic stimulation of either VTA afferent was sufficient to drive syllable-specific learning, and these learning effects were of opposite valence. Finally, functional, anatomical, and molecular studies support the idea that these evaluative afferents bidirectionally modulate VTA dopamine neurons to enable temporally precise vocal learning.

Learning to Expect: Predicting Sounds During Movement Is Related to Sensorimotor Association During Listening

Sensory experiences, such as sound, often result from our motor actions. Over time, repeated sound-producing performance can generate sensorimotor associations. However, it is not clear how sensory and motor information are associated. Here, we explore if sensory prediction is associated with the formation of sensorimotor associations during a learning task. We recorded event-related potentials (ERPs) while participants produced index and little finger-swipes on a bespoke device, generating novel sounds. ERPs were also obtained as participants heard those sounds played back. Peak suppression was compared to assess sensory prediction. Additionally, transcranial magnetic stimulation (TMS) was used during listening to generate finger-motor evoked potentials (MEPs). MEPs were recorded before and after training upon hearing these sounds, and then compared to reveal sensorimotor associations. Finally, we explored the relationship between these components. Results demonstrated that an increased positive-going peak (e.g., P2) and a suppressed negative-going peak (e.g., N2) were recorded during action, revealing some sensory prediction outcomes (P2: p = 0.050, ηp2 = 0.208; N2: p = 0.001, ηp2 = 0.474). Increased MEPs were also observed upon hearing congruent sounds compared with incongruent sounds (i.e., associated to a finger), demonstrating precise sensorimotor associations that were not present before learning (Index finger: p < 0.001, ηp2 = 0.614; Little finger: p < 0.001, ηp2 = 0.529). Consistent with our broad hypotheses, a negative association between the MEPs in one finger during listening and ERPs during performance of the other was observed (Index finger MEPs and Fz N1 action ERPs; r = −0.655, p = 0.003). Overall, data suggest that predictive mechanisms are associated with the fine-tuning of sensorimotor associations.

Auditory-Motor Matching in Vocal Recognition and Imitative Learning

Songbirds possess mirror neurons (MNs) activating during the perception and execution of specific features of songs. These neurons are located in high vocal center (HVC), a premotor nucleus implicated in song perception, production and learning, making worth to inquire their properties and functions in vocal recognition and imitative learning. By integrating a body of brain and behavioral data, we discuss neurophysiology, anatomical, computational properties and possible functions of songbird MNs. We state that the neurophysiological properties of songbird MNs depends on sensorimotor regions that are outside the auditory neural system. Interestingly, songbirds MNs can be the result of the specific type of song representation possessed by some songbird species. At the functional level, we discuss whether songbird MNs are involved in others' song recognition, by dissecting the function of recognition in various different but possible overlapping processes: action-oriented perception, discriminative-oriented perception and identification of the signaler. We conclude that songbird MNs may be involved in recognizing other singer's vocalizations, while their role in imitative learning still require to solve how auditory feedback are used to correct own vocal performance to match the tutor song. Finally, we compare songbird and human mirror responses, hypothesizing a case of convergent evolution, and proposing new experimental directions.

The Avian Basal Ganglia Are a Source of Rapid Behavioral Variation That Enables Vocal Motor Exploration

The basal ganglia (BG) participate in aspects of reinforcement learning that require evaluation and selection of motor programs associated with improved performance. However, whether the BG additionally contribute to behavioral variation ("motor exploration") that forms the substrate for such learning remains unclear. In songbirds, a tractable system for studying BG-dependent skill learning, a role for the BG in generating exploratory variability, has been challenged by the finding that lesions of Area X, the song-specific component of the BG, have no lasting effects on several forms of vocal variability that have been studied. Here we demonstrate that lesions of Area X in adult male zebra finches (Taeniopygia gutatta) permanently eliminate rapid within-syllable variation in fundamental frequency (FF), which can act as motor exploration to enable reinforcement-driven song learning. In addition, we found that this within-syllable variation is elevated in juveniles and in adults singing alone, conditions that have been linked to enhanced song plasticity and elevated neural variability in Area X. Consistent with a model that variability is relayed from Area X, via its cortical target, the lateral magnocellular nucleus of the anterior nidopallium (LMAN), to influence song motor circuitry, we found that lesions of LMAN also eliminate within-syllable variability. Moreover, we found that electrical perturbation of LMAN can drive fluctuations in FF that mimic naturally occurring within-syllable variability. Together, these results demonstrate that the BG are a central source of rapid behavioral variation that can serve as motor exploration for vocal learning.SIGNIFICANCE STATEMENT Many complex motor skills, such as speech, are not innately programmed but are learned gradually through trial and error. Learning involves generating exploratory variability in action ("motor exploration") and evaluating subsequent performance to acquire motor programs that lead to improved performance. Although it is well established that the basal ganglia (BG) process signals relating to action evaluation and selection, whether and how the BG promote exploratory motor variability remain unclear. We investigated this question in songbirds, which learn to produce complex vocalizations through trial and error. In contrast with previous studies that did not find effects of BG lesions on vocal motor variability, we demonstrate that the BG are an essential source of rapid behavioral variation linked to vocal learning.

Physiological identification of cortico-striatal projection neurons for song control in Bengalese finches

The avian song system is a group of brain areas specialized for vocal learning and production of song. A major cortical control area, HVC, projects both to a motor output circuit and to a striatal area in the anterior forebrain pathway. These projections are made by two groups of neurons, with mainly distinct roles in either programming vocal production or regulating vocal plasticity. In order to distinguish these two types of projection neurons in singing birds, we recorded unit activity in HVC of anesthetized birds, while stimulating in the anterior forebrain nucleus Area X. HVC units identified in this way had a distinct spike waveform, with a much longer duration positive peak than an initial negative one. We further found that units with a very similar spike waveform were phasically active during singing, firing at specific points of a limited number of song syllables. These units were also less active when birds only heard their own song, during the same syllables. While similar results from anesthetized and awake recordings have been reported in previous studies, the combination of both types of experiments here may be useful as a basis for identifying HVC neurons projecting to Area X based on their spike waveforms, and aid further study of their role in song learning and control.

Building a state space for song learning

The songbird system has shed light on how the brain produces precisely timed behavioral sequences, and how the brain implements reinforcement learning (RL). RL is a powerful strategy for learning what action to produce in each state, but requires a unique representation of the states involved in the task. Songbird RL circuitry is thought to operate using a representation of each moment within song syllables, consistent with the sparse sequential bursting of neurons in premotor cortical nucleus HVC. However, such sparse sequences are not present in very young birds, which sing highly variable syllables of random lengths. Here, we review and expand upon a model for how the songbird brain could construct latent sequences to support RL, in light of new data elucidating connections between HVC and auditory cortical areas. We hypothesize that learning occurs via four distinct plasticity processes: 1) formation of 'tutor memory' sequences in auditory areas; 2) formation of appropriately-timed latent HVC sequences, seeded by inputs from auditory areas spontaneously replaying the tutor song; 3) strengthening, during spontaneous replay, of connections from HVC to auditory neurons of corresponding timing in the 'tutor memory' sequence, aligning auditory and motor representations for subsequent song evaluation; and 4) strengthening of connections from premotor neurons to motor output neurons that produce the desired sounds, via well-described song RL circuitry.

Neural mechanisms of vocal imitation: The role of sleep replay in shaping mirror neurons

Learning by imitation involves not only perceiving another individual's action to copy it, but also the formation of a memory trace in order to gradually establish a correspondence between the sensory and motor codes, which represent this action through sensorimotor experience. Memory and sensorimotor processes are closely intertwined. Mirror neurons, which fire both when the same action is performed or perceived, have received considerable attention in the context of imitation. An influential view of memory processes considers that the consolidation of newly acquired information or skills involves an active offline reprocessing of memories during sleep within the neuronal networks that were initially used for encoding. Here, we review the recent advances in the field of mirror neurons and offline processes in the songbird. We further propose a theoretical framework that could establish the neurobiological foundations of sensorimotor learning by imitation. We propose that the reactivation of neuronal assemblies during offline periods contributes to the integration of sensory feedback information and the establishment of sensorimotor mirroring activity at the neuronal level.

A Neural Code That Is Isometric to Vocal Output and Correlates with Its Sensory Consequences

What cortical inputs are provided to motor control areas while they drive complex learned behaviors? We study this question in the nucleus interface of the nidopallium (NIf), which is required for normal birdsong production and provides the main source of auditory input to HVC, the driver of adult song. In juvenile and adult zebra finches, we find that spikes in NIf projection neurons precede vocalizations by several tens of milliseconds and are insensitive to distortions of auditory feedback. We identify a local isometry between NIf output and vocalizations: quasi-identical notes produced in different syllables are preceded by highly similar NIf spike patterns. NIf multiunit firing during song precedes responses in auditory cortical neurons by about 50 ms, revealing delayed congruence between NIf spiking and a neural representation of auditory feedback. Our findings suggest that NIf codes for imminent acoustic events within vocal performance.

Transmission of birdsong across generations requires tight interactions between auditory and vocal systems. However, how these interactions take place is poorly understood. We studied neuronal activity in the brain area located at the intersection between auditory and song motor areas, which is known as the nucleus interface of the nidopallium. By recording during singing from neurons in the nucleus interface of the nidopallium that project to motor areas, we found that their spiking precedes peaks in vocal amplitudes by about 50 ms. Notably, quasi-identical notes produced at different times in the song motif were preceded by highly similar spike patterns in these projection neurons. Such local isometry between output from the nucleus interface of the nidopallium and vocalizations suggests that projection neurons in this brain area code for imminent acoustic events within vocal performance. In support of this conclusion, during singing, projection neurons do not respond to playback of white noise sound stimuli, and activity in the nucleus interface of the nidopallium precedes by about 50 ms neural activity in the avian analogue of auditory cortex. Therefore, we conclude that the role of neuronal activity in the nucleus interface of the nidopallium could be to link desired auditory targets to suitable motor commands required for hitting these targets.

Toward an Integration of Deep Learning and Neuroscience

Neuroscience has focused on the detailed implementation of computation, studying neural codes, dynamics and circuits. In machine learning, however, artificial neural networks tend to eschew precisely designed codes, dynamics or circuits in favor of brute force optimization of a cost function, often using simple and relatively uniform initial architectures. Two recent developments have emerged within machine learning that create an opportunity to connect these seemingly divergent perspectives. First, structured architectures are used, including dedicated systems for attention, recursion and various forms of short- and long-term memory storage. Second, cost functions and training procedures have become more complex and are varied across layers and over time. Here we think about the brain in terms of these ideas. We hypothesize that (1) the brain optimizes cost functions, (2) the cost functions are diverse and differ across brain locations and over development, and (3) optimization operates within a pre-structured architecture matched to the computational problems posed by behavior. In support of these hypotheses, we argue that a range of implementations of credit assignment through multiple layers of neurons are compatible with our current knowledge of neural circuitry, and that the brain's specialized systems can be interpreted as enabling efficient optimization for specific problem classes. Such a heterogeneously optimized system, enabled by a series of interacting cost functions, serves to make learning data-efficient and precisely targeted to the needs of the organism. We suggest directions by which neuroscience could seek to refine and test these hypotheses.

A Comprehensive Account of Sound Sequence Imitation in the Songbird

The amazing imitation capabilities of songbirds show that they can memorize sensory sequences and transform them into motor activities which in turn generate the original sound sequences. This suggests that the bird's brain can learn (1) to reliably reproduce spatio-temporal sensory representations and (2) to transform them into corresponding spatio-temporal motor activations by using an inverse mapping. Neither the synaptic mechanisms nor the network architecture enabling these two fundamental aspects of imitation learning are known. We propose an architecture of coupled neuronal modules that mimick areas in the song bird and show that a unique synaptic plasticity mechanism can serve to learn both, sensory sequences in a recurrent neuronal network, as well as an inverse model that transforms the sensory memories into the corresponding motor activations. The proposed membrane potential dependent learning rule together with the architecture that includes basic features of the bird's brain represents the first comprehensive account of bird imitation learning based on spiking neurons.

The Neural Basis of Vocal Pitch Imitation in Humans

Vocal imitation is a phenotype that is unique to humans among all primate species, and so an understanding of its neural basis is critical in explaining the emergence of both speech and song in human evolution. Two principal neural models of vocal imitation have emerged from a consideration of nonhuman animals. One hypothesis suggests that putative mirror neurons in the inferior frontal gyrus pars opercularis of Broca's area may be important for imitation. An alternative hypothesis derived from the study of songbirds suggests that the corticostriate motor pathway performs sensorimotor processes that are specific to vocal imitation. Using fMRI with a sparse event-related sampling design, we investigated the neural basis of vocal imitation in humans by comparing imitative vocal production of pitch sequences with both nonimitative vocal production and pitch discrimination. The strongest difference between these tasks was found in the putamen bilaterally, providing a striking parallel to the role of the analogous region in songbirds. Other areas preferentially activated during imitation included the orofacial motor cortex, Rolandic operculum, and SMA, which together outline the corticostriate motor loop. No differences were seen in the inferior frontal gyrus. The corticostriate system thus appears to be the central pathway for vocal imitation in humans, as predicted from an analogy with songbirds.

Mirror Neurons in a New World Monkey, Common Marmoset

Mirror neurons respond when executing a motor act and when observing others' similar act. So far, mirror neurons have been found only in macaques, humans, and songbirds. To investigate the degree of phylogenetic specialization of mirror neurons during the course of their evolution, we determined whether mirror neurons with similar properties to macaques occur in a New World monkey, the common marmoset (Callithrix jacchus). The ventral premotor cortex (PMv), where mirror neurons have been reported in macaques, is difficult to identify in marmosets, since no sulcal landmarks exist in the frontal cortex. We addressed this problem using "in vivo" connection imaging methods. That is, we first identified cells responsive to others' grasping action in a clear landmark, the superior temporal sulcus (STS), under anesthesia, and injected fluorescent tracers into the region. By fluorescence stereomicroscopy, we identified clusters of labeled cells in the ventrolateral frontal cortex, which were confirmed to be within the ventrolateral frontal cortex including PMv after sacrifice. We next implanted electrodes into the ventrolateral frontal cortex and STS and recorded single/multi-units under an awake condition. As a result, we found neurons in the ventrolateral frontal cortex with characteristic "mirror" properties quite similar to those in macaques. This finding suggests that mirror neurons occur in a common ancestor of New and Old World monkeys and its common properties are preserved during the course of primate evolution.

Naturalistic stimulation drives opposing heterosynaptic plasticity at two inputs to songbird cortex

Songbirds learn precisely sequenced motor skills (songs) subserved by distinct brain areas, including the premotor cortical analog HVC, which is essential for producing learned song, and a 'cortical'-basal ganglia loop required for song plasticity. Inputs from these nuclei converge in RA (robust nucleus of the arcopallium), making it a likely locus for song learning. However, activity-dependent synaptic plasticity has never been described in either input. Using a slice preparation, we found that stimulation patterns based on singing-related activity were able to drive opposing changes in the strength of RA's inputs: when one input was potentiated, the other was depressed, with the direction and magnitude of changes depending on the relative timing of stimulation of the inputs. Moreover, pharmacological manipulations that blocked synaptic plasticity in vitro also prevented reinforcement-driven changes to song in vivo. Together, these findings highlight the importance of precise timing in the basal ganglia-motor cortical interactions subserving adaptive motor skills.

Two neural streams, one voice: pathways for theme and variation in the songbird brain

Birdsong offers a unique model system to understand how a developing brain - once given a set of purely acoustic targets - teaches itself the vocal-tract gestures necessary to imitate those sounds. Like human infants, to juvenile male zebra finches (Taeniopygia guttata) falls the burden of initiating the vocal-motor learning of adult sounds. In both species, adult caregivers provide only a set of sounds to be imitated, with little or no information about the vocal-tract gestures used to produce the sounds. Here, we focus on the central control of birdsong and review the recent discovery that zebra finch song is under dual premotor control. Distinct forebrain pathways for structured (theme) and unstructured (variation) singing not only raise new questions about mechanisms of sensory-motor integration, but also provide a fascinating new research opportunity. A cortical locus for a motor memory of the learned song is now firmly established, meaning that anatomical, physiological, and computational approaches are poised to reveal the neural mechanisms used by the brain to compose the songs of birds.