|
Scientific Understanding of Consciousness |
Dopamine in Stimulus–Reward Learning
Nature 469, 53–57 (06 January 2011) A selective role for dopamine in stimulus–reward learning Molecular and Behavioral Neuroscience Institute, University of Michigan, Michigan, USA Shelly B. Flagel, Leah Mayo, Sarah M. Clinton & Huda Akil Department of Psychiatry and Behavioral Sciences and Department of Pharmacology, University of Washington, Washington, USA Jeremy J. Clark, Ingo Willuhn, Christina A. Akers & Paul E. M. Phillips Department of Psychology, University of Michigan, Michigan, USA Terry E. Robinson & Alayna Czuj [paraphrase] Individuals make choices and prioritize goals using complex processes that assign value to rewards and associated stimuli. During Pavlovian learning, previously neutral stimuli that predict rewards can acquire motivational properties, becoming attractive and desirable incentive stimuli. However, whether a cue acts solely as a predictor of reward, or also serves as an incentive stimulus, differs between individuals. Thus, individuals vary in the degree to which cues bias choice and potentially promote maladaptive behaviour. Here we use rats that differ in the incentive motivational properties they attribute to food cues to probe the role of the neurotransmitter dopamine in stimulus–reward learning. We show that intact dopamine transmission is not required for all forms of learning in which reward cues become effective predictors. Rather, dopamine acts selectively in a form of stimulus–reward learning in which incentive salience is assigned to reward cues. In individuals with a propensity for this form of learning, reward cues come to powerfully motivate and control behaviour. This work provides insight into the neurobiology of a form of stimulus–reward learning that confers increased susceptibility to disorders of impulse control. Dopamine is central for reward-related processes but the exact nature of its role remains controversial. Phasic neurotransmission in the mesolimbic dopamine system is initially triggered by the receipt of reward (unconditional stimulus, US), but shifts to a cue that predicts a reward (conditional stimulus, CS) after associative learning. Dopamine responsiveness appears to encode discrepancies between rewards received and those predicted, consistent with a ‘prediction error’ teaching signal used in formal models of reinforcement learning. Therefore, a popular hypothesis is that dopamine is used to update the predictive value of stimuli during associative learning. In contrast, others have argued that the role of dopamine in reward is in attributing Pavlovian incentive value to cues that signal reward, rendering them desirable in their own right and thereby increasing the pool of positive stimuli that have motivational control over behaviour. Until now it has been difficult to determine whether dopamine mediates the predictive or the motivational properties of reward-associated cues, because these two features are often acquired together. However, the extent to which a predictor of reward acquires incentive value differs between individuals, providing the opportunity to parse the role of dopamine in stimulus–reward learning. Individual variation in behavioural responses to reward-associated stimuli can be seen using one of the simplest reward paradigms, Pavlovian conditioning. If a CS is presented immediately before US delivery at a separate location, some animals approach and engage the CS itself and go to the location of food delivery only upon CS termination. This conditional response (CR), which is maintained by Pavlovian contingency, is called ‘sign-tracking’ because animals are attracted to the cue or sign that indicates impending reward delivery. The core of the nucleus accumbens is an important anatomical substrate for motivated behaviour and has been specifically implicated as a site where dopamine acts to mediate the acquisition and/or performance of Pavlovian conditional approach behaviour. These data provide several lines of evidence demonstrating that dopamine does not act as a universal teaching signal in stimulus–reward learning, but selectively participates in a form of stimulus–reward learning whereby Pavlovian incentive value is attributed to a CS. Here we have shown that stimulus–reward associations that produce different CRs are mediated by different neural circuitry. Previous research using site-specific dopamine antagonism and dopamine-specific lesions indicated that dopamine acts in the nucleus accumbens core to support the learning and performance of sign-tracking behaviour. This work demonstrates that dopamine-encoded prediction-error signals are indeed present in the nucleus accumbens of sign-trackers, but not in the nucleus accumbens of goal-trackers. Although these neurochemical data alone do not rule out the possibility that prediction-error signals are present in other dopamine terminal regions, the results from systemic dopamine antagonism demonstrate that intact dopamine transmission is generally not required for learning of a goal-tracking CR. We thus show that dopamine is an integral part of stimulus–reward learning that is specifically associated with the attribution of incentive salience to reward cues. Individuals who attribute reward cues with incentive salience find it more difficult to resist such cues, a feature associated with reduced impulse control. Human motivated behaviour is subject to a wide span of individual differences ranging from highly deliberative to highly impulsive actions directed towards the acquisition of rewards. This work provides insight into the biological basis of these individual differences, and may provide an important step for understanding and treating impulse-control problems that are prevalent across several psychiatric disorders. [end of paraphrase]
Return to — NeurotransmittersReturn to — Limbic System |