|
Scientific Understanding of Consciousness |
Cortical-Basal Ganglia Circuit for Covert Skill Learning
Nature 486, 251–255 (14 June 2012) Covert skill learning in a cortical-basal ganglia circuit Jonathan D. Charlesworth, Timothy L. Warren & Michael S. Brainard W. M. Keck Center for Integrative Neuroscience, Department of Physiology, and the Neuroscience Graduate Program, University of California, San Francisco, California 94143, USA [paraphrase] We learn complex skills such as speech and dance through a gradual process of trial and error. Cortical-basal ganglia circuits have an important yet unresolved function in this trial-and-error skill learning; influential ‘actor–critic’ models propose that basal ganglia circuits generate a variety of behaviours during training and learn to implement the successful behaviours in their repertoire. Here we show that the anterior forebrain pathway (AFP), a cortical-basal ganglia circuit, contributes to skill learning even when it does not contribute to such ‘exploratory’ variation in behavioural performance during training. Our results suggest a revised model of skill learning: basal ganglia circuits can monitor the consequences of behavioural variation produced by other brain regions and then direct those brain regions to implement more successful behaviours. The ability of the AFP to identify successful performances generated by other brain regions indicates that basal ganglia circuits receive a detailed efference copy of premotor activity in those regions. The capacity of the AFP to implement successful performances that were initially produced by other brain regions indicates precise functional connections between basal ganglia circuits and the motor regions that directly control performance. We assessed the contributions of basal ganglia circuitry to learned modification of adult Bengalese finch song, a complex behaviour consisting of a sequence of 30–100-ms ‘syllables’, each with a highly stereotyped acoustic structure. The song-specific motor control system consists of a motor pathway, which is analogous to mammalian premotor and primary motor cortex and is sufficient to produce well-learned elements of song, and the AFP, which is necessary for juvenile song learning and adult song modification. Our results motivate a revision to models of song plasticity and influential actor–critic models of skill learning, which propose that essential learning-related signals develop only in brain regions that are ‘acting’ (that is, controlling behaviour). In contrast, our results indicate that the essential learning-related signals necessary to adaptively bias behaviour can develop in a basal ganglia circuit, the AFP, while it is prevented from contributing to behavioural performance and motor exploration. This indicates that motor exploration (that is, variation) generated by the AFP is not necessary for learning, and therefore a source of variation independent of the AFP can be exploited for reinforcement learning. Presumably, this variation arises in the motor pathway, possibly in RA, and is transmitted to the AFP. In normal circumstances with AFP output intact, variation contributed by the AFP itself may also be used for reinforcement learning. The AFP may therefore be a specialized hub where information about behavioural variation from multiple sources converges and is associated with reinforcement signals to guide learning. The specificity of learning with AFP output blocked implies that the AFP associates reinforcement signals with detailed information about ongoing song performance, including both the identity of the syllable being produced and the rendition-by-rendition variation in the fundamental frequency of that syllable. Reinforcement signals, indicating the presence or absence of white noise, could be conveyed to the AFP by means of known projections from neuromodulatory nuclei such as the ventral tegmental area. Signals encoding syllable identity are conveyed to the AFP by means of projections from nucleus HVC in the motor pathway to the striatopallidal nucleus Area X. In principle, auditory feedback could provide information about variation in fundamental frequency, but such auditory signals seem to be absent from the AFP during singing. We therefore favour the alternative possibility, that information about fundamental frequency variation is transmitted to the AFP through an efference copy of activity in premotor regions, by way of projections from HVC to Area X and/or projections from RA to the basal ganglia-recipient thalamic nucleus DLM (dorsolateral division of the medial thalamus). This is consistent with a recent proposal that the transmission of efference copy signals from motor cortex (HVC and/or RA) to basal ganglia circuitry (AFP) has a fundamental function in mammalian skill learning. Our results also indicate precise functional coordination between the AFP and the motor pathway. Immediately after unblocking AFP output, we observed learning that was specific to the reinforced features of song, indicating that the AFP had modified its output to direct the production of those specific features by the motor pathway. This implies not only that the AFP receives detailed information about the song performances produced by the motor pathway during training, but also that it changes its output to specifically implement the features of those performances that were reinforced. Such a capacity of the AFP to precisely monitor and modify the activity of the motor pathway indicates fine-scale functional coordination both in the projections from the motor pathway to the AFP and in the projections from the AFP back to the motor pathway. Such bi-directional coordination might be mediated by segregated functional loops between the AFP and the motor pathway, each encoding a particular feature of song, such as high fundamental frequency in a particular syllable. Under normal conditions, with AFP output intact, such functional loops could enable the AFP to amplify and bias specific behavioural features, functions that have been attributed to mammalian basal ganglia circuits. More generally, our results suggest that precise functional coordination between motor cortex and basal ganglia circuitry is important for enabling motor skill learning. [end of paraphrase] |