2010 January 1. Published in final edited form as:
SHELLY B. FLAGEL,* HUDA AKIL,* and TERRY E. ROBINSON†
Drugs of abuse acquire different degrees of control over thoughts and actions based not only on the effects of drugs themselves, but also on predispositions of the individual. Those individuals who become addicted are unable to shift their thoughts and actions away from drugs and drug-associated stimuli. Thus in addicts, exposure to places or things (cues) that have been previously associated with drug-taking often instigates renewed drug-taking. We and others have postulated that drug-associated cues acquire the ability to maintain and instigate drug-taking behavior in part because they acquire incentive motivational properties through Pavlovian (stimulus-stimulus) learning. In the case of compulsive behavioral disorders, including addiction, such cues may be attributed with pathological incentive value (“incentive salience”). For this reason, we have recently begun to explore individual differences in the tendency to attribute incentive salience to cues that predict rewards. When discrete cues are associated with the non-contingent delivery of food or drug rewards some animals come to quickly approach and engage the cue even if it is located at a distance from where the reward will be delivered. In these animals the reward-predictive cue itself becomes attractive, eliciting approach towards it, presumably because it is attributed with incentive salience. Animals that develop this type of conditional response are called “sign-trackers”. Other animals, “goal-trackers”, do not approach the reward-predictive cue, but upon cue presentation they immediately go to the location where food will be delivered (the “goal”). For goal-trackers the reward-predictive cue is not attractive, presumably because it is not attributed with incentive salience. We review here preliminary data suggesting that these individual differences in the tendency to attribute incentive salience to cues predictive of reward may confer vulnerability or resistance to compulsive behavioral disorders, including addiction. It will be important, therefore, to study how environmental, neurobiological and genetic interactions determine the extent to which individuals attribute incentive value to reward-predictive stimuli.
The major clinical problem in the treatment of addiction is the tendency of addicts to relapse even long after the discontinuation of drug use. In fact, the most predictable outcome of a first diagnosis of addiction is a 90% chance of relapse (DeJong, 1994), and this is due in part to sensitivity to drug-associated stimuli. For example, when addicts encounter cues previously associated with drug administration (people, places, paraphernalia, etc.) this often instigates renewed drug-seeking and/or craving for the drug (for review see Childress et al., 1993). One reason drug-related cues are thought to have these effects is because when otherwise neutral environmental stimuli are repeatedly paired with the administration of rewards, including potentially addictive drugs, such stimuli not only act as predictors of impending reward, but through Pavlovian (stimulus-stimulus) learning they acquire incentive motivational properties—they become imbued with “incentive salience” (Berridge, 2001; Bindra, 1978; Bolles, 1972; Toates, 1986). The attribution of incentive salience is defined by Berridge (1996) as the transformation of an otherwise relatively neutral perceptual or representational event (e.g., cue) into an attractive and “wanted” incentive stimulus. That is, through Pavlovian learning sensory information about rewards and their cues (sights, sounds and smells) is transformed into “attractive, desired, riveting incentives” (Berridge and Robinson, 2003). Consequentially, conditional stimuli can become become “motivational magnets” (Berridge, 2001) eliciting approach towards them, as in the case of Pavlovian conditional approach behavior towards rewards and their signals (see below and Cardinal et al., 2002a). Tomie (1996) has argued that when such cues are embedded in the device that delivers a drug, such as specialized glassware used to consume alcohol or pipes used to smoke cocaine, these cues can promote especially compulsive patterns of behavior characteristic of addiction.
It is thought that the incentive motivational properties of Pavlovian cues, their ability to goad an individual into action, are primarily responsible for the ability of drug-associated cues (and places) both to maintain drug-seeking in the absence of the drug (Everitt and Robbins, 2000; Schindler et al., 2002) and to reinstate drug-seeking behavior or relapse during abstinence (e.g., Kruzich et al., 2001). However, we would be remiss if we did not mention that there are other ways that cues (and especially contexts) associated with drugs can influence drug-seeking behavior and relapse. For example, drug-associated cues can act as discriminative stimuli, “setting the occasion” for drug-taking behavior (Skinner, 1938). Occasion-setters are conditional cues or contexts that may not themselves elicit a response or reinforce an action, but act in a hierarchical manner to modulate either instrumental responding to obtain a reward and/or the occurrence of Pavlovian conditional responses (Holland, 1992; Rescorla, 1988; Schmajuk and Holland, 1998). Such stimuli can have a powerful influence on drug-seeking behavior. For example, (Ciccocioppo et al., 2004) reported that a discriminative stimulus associated with a single cocaine self-administration session can elicit robust cocaine-seeking behavior up to one year after the last experience with cocaine. Similarly, contexts often serve to set the occasion for renewed drug-seeking (Crombag and Shaham, 2002) and can even modulate the expression of behavioral sensitization (Anagnostaras and Robinson, 1996; Anagnostaras et al., 2002). Of course, exposure to a drug itself (a drug prime) may be effective in producing relapse/reinstatement in part because the interoceptive cues produced by the drug can also “set the occasion” for further drug-seeking, although this begs the question as to what psychological processes led to the first drink, puff or injection. The extent to which these “higher-order” conditional stimuli act as incentive stimuli is not well understood.
Many theories of addiction have recognized the role of Pavlovian conditional motivation in attracting humans towards drug-associated stimuli and sources of addictive drugs, and in relapse (Di Chiara, 1998; Everitt and Robbins, 2005; Robinson and Berridge, 1993; Stewart et al., 1984; Tomie, 1996; Tomie et al., 2007), yet there have been very few attempts to directly study this process. In one example See and colleagues (Kruzich et al., 2001) reported that classically conditioned cues can reinstate drug-seeking behavior. But in most pre-clinical studies on how cues influence drug-seeking and -taking, the initial association between a cue and drug delivery occurs in an instrumental learning setting, where both the cue and drug are presented contingent upon an action. In these studies, for example, an animal will be trained to lever press in order to receive a drug infusion, which is accompanied by the presentation of a cue (e.g., light). Cue-induced reinstatement is later studied under extinction conditions (when the drug is no longer available), and after responding subsides the ability of the drug-associated cue (e.g., light) to reinstate drug-seeking behavior is examined. Thus, during tests of reinstatement the action of lever pressing behavior, that previously produced the drug and light, now results in presentation of the light alone (for review see Shaham et al., 2003). The cue, therefore, can act as a conditional (secondary) reinforcer, reinforcing the action that produces it. In humans, however, environmental cues associated with injection of a drug are usually present prior to injecting the drug—they do not suddenly appear as a consequence of taking the drug. Moreover, in addicts, cues that produce relapse or craving generally precede actions to acquire the drug, they do not follow them; i.e., they act as incentive stimuli goading the individual to action rather than reinforcing an action that has already been emitted (Stewart et al., 1984).
There is, therefore, a dearth of information on the effects of Pavlovian conditional cues on drug-seeking or drug-taking behavior. Most studies in the field involve procedures that make it very difficult to determine the extent to which behavior is controlled by instrumental reinforcement (reinforcing an action that has already occurred) vs. Pavlovian incentives that instigate actions. This distinction is important because the psychological and neurobiological processes underlying instrumental learning may be very different from Pavlovian processes that confer incentive value upon drug-associated cues and places (Cardinal et al., 2002a; Tomie et al., 2007). Thus, in trying to understand how drug cues can maintain drug-seeking behavior for long periods of time even in the absence of the drug, and how they can precipitate relapse, we need to parse out the basic psychological processes by which such stimuli acquire incentive salience (Berridge and Robinson, 2003), and in turn delineate the underlying neural substrates. For this reason we have recently begun to explore the ability of Pavlovian cues to control behavior (Flagel et al., 2008; Flagel et al., 2007; Uslaner et al., 2006), and especially individual differences in how such cues influence behavior. This work is preliminary, but we hope it will eventually provide insights into the psychological and neurobiological mechanisms by which drug-related cues gain the ability to control behavior in addiction. In the following we first discuss some of the historical literature on how researchers have studied the attribution of incentive salience to reward-related cues, and how this is revealed in behavior, and then our recent studies on individual differences.
In classical Pavlovian conditioning the presentation of a cue (conditional stimulus, CS) is associated with the presentation of a reward (the unconditional stimulus, US), and with repeated pairings the CS comes to elicit a conditional response (CR). In Pavlov’s original studies using a food reward his dogs were typically restrained and the response to the CS often measured was salivation, giving the appearance that the CS evoked a simple reflexive response similar to the unconditional response (UR) produced by the US. However, when animals were freed from their restraints and their behavior observed, it became obvious that what was learned was not just a simple reflexive response, because in this situation the CS evoked complex patterns of behavior, including food-begging behavior (H. Liddell, unpublished, cited in Timberlake and Grant, 1975). Many subsequent studies have shown that in addition to simple responses, Pavlovian CSs evoke complex emotional and motivational states that can be manifest in a variety of ways (Rescorla, 1988). Thus, when animals are free to move in their environment it is often observed that the CR consists of initial orientation to the CS (the cue or “sign”) followed by approach towards it, and frequently engagement with it (Brown and Jenkins, 1968; Hearst and Jenkins, 1974). Collectively, these CS-directed responses were called “sign tracking”, because behaviors were directed towards the cue, or “sign”, that predicted reward. Remarkably, sign tracking develops even though no action is required for the animals to receive the reward; that is, no actions are reinforced. (Note that the term “autoshaping” is often used to describe the procedure that produces this form of Pavlovian conditional approach behavior, although “autoshaping” is really a misnomer in that in the Pavlovian procedure no response is reinforced or “shaped”; Hearst and Jenkins, 1974).
With autoshaping the exact topography of the CR depends on the species, and the nature of both the CS and the US. It was originally noted that there is often a striking similarity between the behavioral patterns involved in consuming a reward and those directed towards the CS. In his studies with dogs and a food reward Pavlov (1932) stated, “…the animal may lick the electric lamp [CS], or appear to take the air into its mouth, or to eat the sound…licking his lips and making the noise of chewing with his teeth as though it were a matter of having the food itself” (pg. 95). Similarly, if presentation of a lever is immediately followed by the response-independent delivery of a food pellet, some rats will approach and often grasp and gnaw the lever as if it were itself food (Davey and Cleland, 1982). And when pigeons are exposed to a key-light (CS) that has been paired with presentation of water (US) they exhibit a drinking-specific motor pattern (complete with gullet movement) directed at the key-light. This pattern of behavior is distinct from the response that emerges in pigeons following pairings with food reward (Jenkins and Moore, 1973; see also http://go.owu.edu/~deswartz/introduction.html). Remarkably, when a CS is paired with the opportunity to copulate with a female (US), male Japanese quail (Coturnix japonica) will, under some conditions, come to approach and copulate with the inanimate object CS (Koksal et al., 2004, see also Burns and Domjan, 1996; Burns and Domjan, 2001).
The fact that presentation of the CS often leads to a CR that resembles behavior elicited by the reward itself (e.g., consummatory behavior) led to the notion that the CS acts as a surrogate for the US—“stimulus-substitution” (Pavlov, 1927; Staddon and Simmelhag, 1971). Although this is consistent with the theory that the CS takes on the incentive properties of the reward, in the past some have argued that such behavior is a mere reflection of sensorimotor conditioning in the absence of any motivational processes (see Berridge, 2001). However, we now know that CSs that evoke these consummatory reactions acquire the three fundamental properties of incentive stimuli (Berridge, 2001; Cardinal et al., 2002a): (1) they have to ability to elicit approach towards them as indicated by Pavlovian conditional approach or sign-tracking behavior (e.g., Flagel et al., 2008; Hearst and Jenkins, 1974; Peterson et al., 1972); (2) they can energize ongoing instrumental actions, as indicated by the Pavlovian-to-instrumental transfer effect (e.g., Dickinson et al., 2000; Lovibond, 1983; Wyvell and Berridge, 2000); and (3) they can reinforce the learning of new instrumental actions; that is, they can act as conditional reinforcers (e.g., Di Ciano and Everitt, 2004; Williams and Dunn, 1991). Moreover, there are many examples illustrating that the form of the CR is influenced by the nature of the CS (see Holland, 1977), suggesting that sign-tracking behavior is not due to simple stimulus-substitution. For example, the form of the CR to a CS that predicts food is very different if the CS is a lever, or a live rat, or a block of wood (Timberlake and Grant, 1975). Thus, the emergent CR is thought to reflect the activation of complex motivational processes that are under the control of a number of factors, including the nature of the US and the CS (Buzsaki, 1982; Davey et al., 1984; Jenkins and Moore, 1973; Moore, 1973; Timberlake and Lucas, 1985).
The phenomenon of sign-tracking has been well characterized with natural rewards used as the US (for review see Boakes, 1977; Hearst and Jenkins, 1974; Tomie et al., 1989) and there are many examples where reward- related cues become so irresistibly attractive that they engender seemingly maladaptive and arguably compulsive behavior (Boakes et al., 1978; Breland and Breland, 1961; Hearst and Jenkins, 1974; Williams and Williams, 1969). In a classic example, termed the “misbehavior of organisms” (Breland and Breland, 1961, 1966), raccoons were trained to deposit a wooden coin through a slot in order to obtain a food reward. The raccoons initially performed this task without hesitation, but with further training seemed unable to let go of the coin, spending several minutes compulsively handling it with their forepaws—chewing, licking, rubbing and washing the coin—as if they were trying to clean a morsel of food —and repeatedly putting the coin into the slot but then pulling it back out without releasing it. The coin itself appeared to have great incentive value, as the raccoons were very reluctant to give it up, even though holding onto it delayed or even prevented receipt of actual food. Another interesting example, termed “negative automaintenance”, was originally described by Hearst and Jenkins (1974), who paired illumination of a key light at one end of a long box with subsequent delivery of a food reward at the other end. Although no response was required to receive the food, pigeons began to approach and repeatedly peck the key light, even though doing so prevented them from retrieving the food, which was available at the other end of the box for only a limited amount of time. This behavior persisted despite the fact that the subjects were often quite hungry. One would expect that if the pigeons were able to quit responding to the distant CS (and consequentially retrieve and consume the reward) they would do so. This apparently compulsive pattern of responding is similar to that seen in addiction in that it is triggered by an impulse determined by past experience and seems to be relatively independent of volition (Tomie, 1996). Moreover, the lack of inhibitory control (i.e., maladaptive behavior) apparent in animals that sign-track is a defining attribute of impulsivity and further links this behavior with addictive behaviors and other impulse control disorders (Tomie, 1996; Tomie et al., 1998).
The studies described above are in agreement with a number of others that have shown that sign-tracking is not due to “accidental reinforcement” of the response and that is persists even when it leads to loss of reinforcement (see Gamzu and Williams, 1971; Killeen, 2003; Lajoie and Bindra, 1976; Timberlake and Lucas, 1985). Despite these findings, some researchers have questioned the role of Pavlovian (stimulus-stimulus) processes in sign-tracking behavior and have claimed that such behavior may be due to response reinforcement (e.g., see Farwell and Ayres, 1979; Locurto et al., 1976; Locurto, 1981; Myerson et al., 1979; Sanabria et al., 2006; Wessels, 1974). For example, after autoshaping training whereby rats developed a sign-tracking CR, Locurto and colleagues (1976) transferred the animals to an omission schedule. On this schedule if the animals contacted the CS-lever the food reward was withheld. Previous studies suggested that in this situation pigeons continue to contact a CS-key light, although at a lower rate, which was taken as evidence that the behavior was not controlled by response reinforcement (e.g., Williams and Williams, 1969). In contrast, Locurto et al. (1976) reported that under these conditions rats stopped contacting the CS-lever, which they took to indicate that the behavior was mediated by response reinforcement. However, even in this latter study there is clear evidence that the CS-lever maintained its Pavlovian conditional motivational properties because Locurto and colleagues (1976) also reported that the topography of the conditional response differed depending on the schedule of reinforcement (i.e., autoshaping vs. omission). Prior to instituting the omission schedule the CR consisted of approach to the lever and contact with it (i.e., gnawing and biting it). During the omission schedule the animals continued to approach and exhibit investigatory behavior (i.e., sniffing) directed towards the CS-lever, but simply stopped physically contacting it. These findings suggest that even under omission conditions the CS-lever continued to have incentive properties—the rats continued to approach and direct their attention to it, and what they learned was to avoid directly contacting it. They withheld only the terminal link in the chain of behaviors that comprised the Pavlovian CR (see also Stiers and Silberberg, 1974). Thus, even these data are consistent with the conclusion that a sign-tracking CR (approach) is dependent upon Pavlovian learning and not response reinforcement, and further illustrate the ability of Pavlovian cues to maintain behavior even in the absence of response reinforcement.
Sign-tracking to drug-associated cues
Tomie (1996) was amongst the first to describe the similarities between Pavlovian sign-tracking behavior and symptoms of drug abuse, and in recent years drug abuse researchers have begun to further explore this relationship (Flagel et al., 2008; Flagel et al., 2006; Newlin, 2002; Tomie et al., 2007). There are, however, still very few studies on the extent to which drug-associated cues support a sign-tracking CR (Cunningham and Patel, 2007; Kearns et al., 2006; Krank et al., 2008; Uslaner et al., 2006). Indeed, in a 2005 review Everitt and Robbins duly noted (pg. 1482), “it might logically be thought that Pavlovian approach is involved in maladaptively attracting humans toward sources of addictive drug reinforcers…as emphasized in the incentive salience theory of addiction. However…approach to a CS predictive of a drug…has [not] been clearly demonstrated in laboratory studies…although… [it] is readily seen in animals responding for natural rewards. It may be…that the behavioral influence of CSs associated with drugs and natural reinforcers differ fundamentally in this regard”. This idea was initially supported by a report that rats do not approach discrete cues paired with i.v. cocaine delivery (Kearns and Weiss, 2004). If it is true that “drugs and natural reinforcers differ fundamentally in this regard” a number of theories of addiction would require serious revision.
However, there have now been a number of reports that animals do approach discrete cues that have been associated with drug delivery (Cunningham and Patel, 2007; Krank et al., 2008; Uslaner et al., 2006). For example, a US consisting of an orally consumed ethanol/saccharin (Krank, 2003; Tomie, 2001; Tomie et al., 2003) or amphetamine/saccharin solution (Tomie, 2001) can support sign-tracking behavior. One concern with these studies is the possibility that it was the sweet solution rather than the alcohol that supported approach, although there were attempts to control for this variable. More recently, Krank and colleagues (2008) unambiguously demonstrated the ability of an ethanol-paired cue (light) to elicit sign-tracking behavior in rats using unsweetened ethanol solution as the US. In addition, Pavlovian conditional approach to a visual cue associated with intraperitoneal injections of ethanol has recently been demonstrated in mice using a modified conditioned place preference procedure thought to reflect sign-tracking behavior (Cunningham and Patel, 2007). Finally, an ethanol-associated cue has also been shown to produce Pavlovian-instrumental transfer effects (Corbit and Janak, 2007).
To our knowledge the first study to demonstrate that the i.v. administration of a drug could support sign-tracking to a discrete cue was by Uslaner et al. (2006). In this experiment an 8 sec presentation of an illuminated retractable lever (CS) was paired with the response-independent delivery of an intravenous injection of cocaine (US). Eight CS-US pairings were presented on a schedule with a randomly varying inter-trial interval with a mean of 900 sec. With repeated pairings of the lever and cocaine rats began to approach the lever more reliably and more rapidly, whereas rats that received pseudorandom (i.e., unpaired) CS-US presentations did not. These findings are in contrast to those reported by Kearns and Weiss (2004), and we speculate that these disparate results are due to methodological differences (for details see Uslaner et al., 2006). Kearns and Weiss (2004) used relatively short inter-trial intervals, with CS-US presentations occurring on average every 90 sec. In contrast to food reward, neurobiological and interoceptive effects of cocaine persist for longer than 90 sec. It is therefore possible that the effects of the previous drug infusion were still being experienced upon subsequent CS-US pairings, making it difficult for the rats to associate these events. Indeed, Uslaner and colleagues (2006) successfully used the longer inter-trial interval of 900 sec only after failing to observe sign-tracking behavior with shorter inter-trial intervals.
In an independent (unpublished) study using selectively-bred rat lines (Stead et al., 2006), we have recently found that animals that sign-track to a CS associated with food reward also sign-track to a cocaine-paired CS; whereas goal-trackers do not. These findings suggest that individual differences in the control over behavior by cues predictive of food rewards may also apply to drug rewards. It is important to note, however, that the topography of the CR that emerges is quite different when cocaine vs. food is used as the CS. When cocaine is used as the US animals rarely come into contact with the lever, but exhibit clear sign-tracking behavior consisting of orientation to the lever followed by approach and exploration in the immediate vicinity of the extended lever (Uslaner et al., 2006), similar to the CR seen when intracranial electrical stimulation is used as the US (Peterson, 1975; Peterson et al., 1972; Phillips et al., 1981; Wilkie and McDonald, 1978).
Conditioned place preference vs. sign-tracking
Although there are very few studies using autoshaping procedures as a way to study the motivational properties of Pavlovian conditional stimuli, there are, of course, many studies using a different procedure—conditioned place preference (CPP). In this situation non-contingent drug administration is paired with placement into a specific context (place), and saline with another, and on a test day animals have access to both places. Animals are said to show a CPP if they spend more time in the drug-paired context (for review see Bardo and Bevins, 2000). A conditioned place preference is often interpreted as a form of Pavlovian conditional approach behavior in that the drug-paired environment is thought to have acquired incentive motivational properties and become attractive, thus animals are “drawn” to it (Carr et al., 1989). A CPP has even been described as “an awkward form of autoshaping” (Newlin, 1992). However, the nature of the psychological process that leads to a CPP is ambiguous (see Cunningham et al., 2006). Whilst the initial learning in CPP may involve Pavlovian processes, behavior on the test day may not be reflective of Pavlovian conditional approach (McAlonan et al., 1993; White, 1996; White et al., 2005). For example, Cunningham et al. (2006) obtained a CPP using tactile stimuli in the dark, and argued the CPP was probably not due to Pavlovian conditional approach because animals could not detect the relevant stimulus from a distance and thereby be attracted to it. They argued that the CPP was due to conditional reinforcement. Thus, given the ambiguity in interpreting a CPP a more appropriate procedure for studying the “attractiveness” of Pavlovian conditional stimuli may be autoshaping. In the remainder of this paper we will focus on studies from our own laboratory, and others, which have uncovered individual differences in the extent to which Pavlovian conditional stimuli acquire incentive motivational properties, which we hypothesize may contribute to addiction vulnerability.
Individual differences in the attribution of incentive salience to reward-related cues: sign-tracking vs. goal-tracking
As mentioned above, when a discrete cue (CS) is repeatedly presented in association with delivery of a reward (US) a number of different responses can emerge that are conditional upon this relationship (CRs), and the topography of the CR is dependent on a number of factors, including the species or strain of the animal (Boakes, 1977; Kearns et al., 2006; Kemenes and Benjamin, 1989; Nilsson et al., 2008; Purdy et al., 1999), the nature of the CS and US (Burns and Domjan, 1996; Davey and Cleland, 1982; Davey et al., 1984; Domjan et al., 1988; Holland, 1977; Jenkins and Moore, 1973; Peterson et al., 1972; Schwam and Gamzu, 1975; Timberlake and Grant, 1975; Wasserman, 1973), the spatial and temporal contingencies between the CS and US (Brown et al., 1993; Costa and Boakes, 2007; Holland, 1980a; Silva et al., 1992; Timberlake and Lucas, 1985), prior experience with the CS or US (Boughner and Papini, 2003; Engberg et al., 1972; Gamzu and Williams, 1971) and the internal drive state of the animal (Berridge, 2001; Boakes, 1977; Davey and Cleland, 1982; Toates, 1986; Zamble et al., 1985; Zener, 1937). Taken together, all of these factors contribute to the incentive motivational state of the animal and thus affect the degree to which the CS acquires incentive motivational properties (Lajoie and Bindra, 1976).
However, even when trained under identical conditions there can be large individual differences in the nature of the CR. To our knowledge, Zener (1937) was the first to systematically describe such individual differences when animals were trained using classic Pavlovian conditioning procedures. Zener (1937) paired ringing a bell with food delivery, but in contrast to most of Pavlov’s (1932) original experiments the dogs in Zener’s studies were not restrained. This not only revealed the complexity of the CRs that develop, but also large individual differences. Zener (1937) reported that after training some dogs responded to the CS with “an initial glance at the bell” followed by “a constant fixation…to the food pan…”, whereas others exhibited a “small but definite movement of approach toward the conditioned stimulus…followed by a backing up later to a position to eat”, a sequence described by Zener as a “striking phenomenon” (pg. 391). Still other dogs vacillated, looking back and forth between the bell and the food pan.
Individual variation in the topography of CRs evoked by a stimulus predictive of food did not receive much further attention until 1977 when Boakes described similar individual differences in rats. Boakes (1977) used a standard autoshaping procedure consisting of brief illumination of a lever immediately followed by the non-contingent delivery of a food pellet. He reported that during CS presentation some rats approached and contacted the lever and went to the food tray after CS offset, similar to what had been described previously (Hearst and Jenkins, 1974). However, upon CS presentation other rats immediately went to the place where food would be delivered, and during the CS period repeatedly operated the tray flap. Boakes (1977), like his predecessors (Hearst and Jenkins, 1974), called CS-evoked approach to the cue (or sign) “sign-tracking” and, for the sake of symmetry, referred to CS-evoked approach to the location where the food would be delivered “goal-tracking,” and we will continue to use this terminology here. Interestingly, the probability of either sign-tracking or goal-tracking was conditional upon the presence of the illuminated lever (CS) and decreased when food (US) was removed or was presented randomly with respect to the CS (Boakes, 1977). Boakes (1977) also found that changes in the probability of reinforcement could significantly affect which behavioral response emerged. Delivering food on only half of the trials was more effective in producing a sign-tracking CR than if food was delivered on every trial, which increased the goal-tracking CR. Altering the level of food deprivation of the subjects had a lesser, but measurable effect (Boakes, 1977).
Given the importance ascribed to Pavlovian conditional processes in controlling behavior, and in addiction—the topic has been the subject of numerous studies and reviews (e.g., Cardinal et al., 2002a; Day and Carelli, 2007; Tomie, 1996; Tomie et al., 2007)—it is surprising there have been very few studies directly comparing sign-tracking vs. goal-tracking CRs since the initial studies by Zener (1937) and Boakes (1977). One explanation for this may be the fact that in most studies of Pavlovian conditional approach behavior the conditions used either do not allow the simultaneous assessment of sign-tracking and goal-tracking, or favor the development of only one of the two CRs (for exceptions see Burns and Domjan, 1996; Nilsson et al., 2008). There have, however, been a few studies on the conditions that favor a sign-tracking vs. goal-tracking CR. For example, using rats, Davey, Cleland and colleagues (Cleland and Davey, 1983; Davey and Cleland, 1982; Davey et al., 1981) reported that sign-tracking occurs to a discrete visual stimulus paired with food reward, but not to a localizable auditory CS. The predominant conditional response to an auditory stimulus is goal-tracking, with little approach behavior directed towards the CS. It has also been established that increasing the temporal or spatial distance between the CS and US results in a decline in the sign-tracking response and an increase in the goal-tracking response (Brown et al., 1993; Costa and Boakes, 2007; Holland, 1980a, b; Silva et al., 1992).
Although there has been some research on the conditions that lead to the development of a sign-tracking vs. a goal-tracking CR, there has been almost no attention paid to individual differences in the propensity to develop one or the other CR. We think this may be an important area for study because it provides a way to evaluate individual differences in the propensity to attribute incentive value to cues that are predictive of rewards. Tomie et al. (2000) has reported individual differences in lever press CR performance (i.e., propensity to sign-track) using an autoshaping paradigm and found associated changes in stress-induced corticosterone release and mesolimbic levels of monoamines. Specifically, Tomie (2000) found those individuals that exhibited an increased sign-tracking response also had elevated levels of post-session corticosterone (see also Figure 4 below) and higher tissue levels of dopamine and DOPAC in the nucleus accumbens, lower levels of DOPAC/DA turnover in the caudate putamen, and lower levels of 5-HIAA and 5-HIAA/5-HT turnover in the ventral tegmental area (VTA). Tomie and colleagues (2000) concluded that the neurochemical profile evident in rats that were more likely to sign-track shared some features of vulnerability to drug abuse. Tomie (1998) has also reported that rats that perform more vigorous sign-tracking behavior are more likely to be impulsive, as measured by the tendency to choose small immediate rewards over larger delayed rewards.
Tomie’s work on sign-tracking to food and ethanol-related cues, and how this behavior may contribute to the development of compulsive drug use, prompted us to begin to explore individual differences in this behavior. Our initial studies focused on developing procedures to study conditional approach to a cue paired with the intravenous delivery of cocaine (Uslaner et al., 2006). However, in doing so we also conducted studies using food as a reward, and in these studies we observed large individual differences in the CR that emerged after pairing presentation of a lever with delivery of a food pellet (Flagel et al., 2008; Flagel et al., 2007), as had been described by Boakes (1977). Briefly, in our studies an illuminated lever (CS) is presented for 8 sec, and immediately after it is retracted a banana-flavored food pellet (US) is delivered into a nearby receptacle located approximately 2.5 cm from the location of the CS-lever. The delivery of the reward is independent of the animal’s behavior (i.e., no explicit action is reinforced). A single session consists of 25 CS-US pairings presented on a random interval 90 sec schedule, and one session (≈40 min) is conducted per day. Rats are not food deprived yet they all consume all of the food pellets that are delivered. Under these conditions, using a sample of commercially available Sprague-Dawley rats, we have found that animals can be divided roughly into three groups based on the conditional response that emerges. Approximately one third of the rats (sign-trackers) come to preferentially approach and vigorously engage the reward-predictive cue (CS-lever; see Figures 1A, 2A, 2C and and3).3). One index of this sign-tracking response is the number of times a rat depresses the lever (Figure 2C) which it typically does by grasping and gnawing it (Figure 1A). When the lever is retracted these animals immediately go and retrieve the food pellet. Approximately another third of the rats (goal-trackers) very rarely approach the reward-predictive CS (Figure 3), but instead, upon presentation of the CS-lever they learn to quickly go to the food receptacle and they come to repeatedly put their nose into it while awaiting delivery of the reward (see Figures 1B, 2B and 2D). The remaining one third of rats (intermediate group) show neither clear sign-tracking nor goal-tracking, but vacillate between the cue and the goal (see Figures 2 and and3;3; Flagel et al., 2008; see also Boakes, 1977; Zener, 1937). All of the animals, regardless of their CR, retrieve and eat all of the food pellets and their behavior during the intertrial intervals is virtually identical. Thus, both sign-tracking and goal-tracking are conditional upon presentation of the lever (the CS). Furthermore, if CS-lever presentation is explicitly not paired with food delivery, but occurs randomly, neither the sign-tracking nor the goal-tracking CR develop (unpublished data).
Although individual differences are apparent even after the first day of training, it is important to emphasize that both sign-tracking and goal-tracking are learned responses that are conditional upon repeated CS-US pairings, and both CRs are evoked by the same cue; that is, the same conditional stimulus (the lever; see Figures 2 and and3).3). For both sign-trackers and goal-trackers repeated CS-US pairings increase the probability of approach during the CS period to either the cue or the goal, respectively (Figures 2 and and3),3), the vigor with which they engage the cue or the goal (Figure 2C and 2D), and the rapidity with which they approach the cue or the goal (Figure 2E and 2F). Furthermore, the tendency of sign-trackers to approach the reward-predictive lever and for the goal-trackers to approach the food cup (goal) during the CS period remains stable, even with prolonged training. In our published reports (Flagel et al., 2008; Flagel et al., 2007) we show data from only 5 days of training, but in Figure 3 we show previously unpublished data in which animals were trained for 16 consecutive days and it is clear that once developed, the sign-tracking and goal-tracking CRs remained stable. In contrast, with extended training, rats in the intermediate group tend to start to sign-track, although still not at levels equivalent to the animals initially classified as sign-trackers. The initial preference for the food cup in all animals (or lack of initial preference for the lever in sign-trackers) is not surprising given that all of the animals were “pre-trained” to retrieve pellets from the food cup (see also Flagel et al., 2008).
Based on these data it is clear that for both sign-trackers and goal-trackers the reward-predictive cue (CS) provides the information necessary to support learning. In both sign-trackers and goal-trackers repeated CS-US pairings result in the development of a learned response that is conditional upon presentation of the CS. That is, the CS is equally predictive of reward for both groups, and both groups develop a conditional response, but what differs is the direction of the CR. Sign-trackers approach and manipulate the reward-predictive cue (lever) itself, even though it is located away from the location where reward will be delivered (food cup). In contrast, upon presentation of the CS-lever goal-trackers do not approach it, but treat it as a signal for impending reward delivery and approach the location where the reward will be delivered (i.e., food cup). Thus, a cue that acts as a predictive, “informational” CS and supports learning in all of the animals becomes “attractive” and elicits approach towards itself in some but not others.
Although the underlying mechanisms have not yet been delineated, it may be useful to consider these findings using the theoretical framework of incentive motivation (Berridge, 2001; Bindra, 1978; Bolles, 1972). As mentioned above, CSs that acquire incentive motivational properties, or using the terminology of Berridge (2001), stimuli that are attributed with incentive salience, become attractive, can energize ongoing behavior and can act as conditional reinforcers (also see Cardinal et al., 2002a). In sign-trackers the CS does become attractive, as indicated by the fact that they approach it. Importantly, we have also found in a series of unpublished studies that for sign-trackers the CS-lever acts as an effective conditional reinforcer, reinforcing the learning of a new instrumental response. Thus, for sign-trackers the CS itself comes to acquire at least two of the defining properties of an incentive stimulus, suggesting that in these animals the CS-lever is attributed with incentive salience. On the other hand, in goal-trackers the CS does not itself appear to be attractive, because they do not approach it, and importantly, in unpublished studies we have found that in goal-trackers the CS-lever is relatively ineffective as a conditional reinforcer. Thus, for goal-trackers the CS does not acquire these defining properties of an incentive stimulus, suggesting that in these animals the CS itself is not attributed with incentive salience.
In the context of this theoretical formulation it is interesting to think about the differences between sign-trackers and goal-trackers. It may be that they do not differ in their ability to attribute incentive salience to stimuli per se, but in where it is attributed. Thus, for goal-trackers it may be that the CS directs incentive salience away from itself—and in this case to the food cup—as goal-trackers come to vigorously engage the food cup and direct consummatory-like behaviors towards it (Mahler and Berridge 2008, personal communication). On the other hand, it is also possible that for goal-trackers the CS evokes a cognitive expectation of reward and their behavior is governed more by cognitive than by incentive processes (Toates, 1998). Whatever the case, taken together these data suggest that in sign-trackers the reward-predictive cue itself, the CS, acquires incentive motivational properties and in goal-trackers it does not. Another interesting implication of these individual differences is that they suggest it is possible to dissociate the predictive or “informational” value of a CS from its incentive motivational properties. Given that the “informational” and “incentive” properties of reward-predictive cues are dissociable psychological phenomena they are presumably mediated by dissociable neural systems. This distinction may be important in exploring the neural mechanisms by which reward-related cues come to control behavior and awaits further research.
Correlates of sign-tracking and goal-tracking behavior
Response to cocaine
Individual differences in the tendency to attribute incentive salience to conditional stimuli are presumably related to individual differences in the organization and operation of brain regions normally involved in this psychological process. The neural system thought to be critically involved includes dopaminergic projections from the VTA to the core of the nucleus accumbens and related circuitry, because there are many studies showing that lesions or pharmacological manipulations of this circuitry prevent the acquisition and expression of Pavlovian conditional approach behavior (Cardinal et al., 2002b; Dalley et al., 2005; Day and Carelli, 2007; Day et al., 2006; Phillips et al., 1981). Therefore, in our initial studies we examined whether sign-trackers and goal-trackers differed in responsiveness to a pharmacological agent that activates dopaminergic systems and is also potentially addictive—cocaine. We examined both the acute psychomotor response to cocaine and the propensity for cocaine-induced psychomotor sensitization (Flagel et al., 2008). Rats first underwent Pavlovian training with food reward as described above to determine their phenotype—sign-tracker or goal-tracker—and were then treated with cocaine. Goal-trackers were moderately more sensitive to the locomotor activating effects of cocaine the first time they experienced the drug (there were group differences at one of three doses tested), although there were no group differences in the ability of cocaine to produce repetitive head movements following acute treatment. In contrast, sign-trackers showed a greater propensity for psychomotor sensitization upon repeated treatment (Flagel et al., 2008). Thus, individual differences in the tendency to sign-track or goal-track are associated with susceptibility to a form of cocaine-induced plasticity—sensitization—that may contribute to the development of addiction. Differences in experience-dependent changes in dopamine function may contribute to the propensity for psychomotor sensitization in these animals (see below).
Although it has been well-documented that dopamine activity in the nucleus accumbens (especially the core) is critical for the development of Pavlovian conditional approach behavior (for review see Cardinal and Everitt, 2004; Day and Carelli, 2007; Everitt and Robbins, 2005; Tomie et al., 2007), we are the first, to our knowledge, to examine the neurobiological correlates of sign-tracking vs. goal-tracking behavior. We have used in situ hybridization to study a number of dopamine-related genes (see Flagel et al., 2007). First, mRNA expression levels were determined when brains were obtained immediately after the first (≈40 min) training session, in the hopes that this would be too soon after the experience to alter the expression of dopamine receptor (D1R or D2R), dopamine transporter (DAT) or tyrosine hydroxylase (TH) mRNA, and thus provide a measure of “basal” expression. In this situation sign-trackers had significantly higher levels of dopamine D1R mRNA in the nucleus accumbens relative to goal-trackers. These findings are in agreement with those of Dalley and colleagues (2005), who reported that accumbens D1R are necessary for the early consolidation and acquisition of sign-tracking behavior. There were no group differences in accumbens D2R mRNA, or in DAT or TH mRNA levels in the VTA after the first training session. Brains were obtained from a separate group of animals after 5 days of training and the development of a strong sign-tracking or goal-tracking CR. These may be more reflective of changes in gene expression that occur as a consequence of learning. After training, sign-trackers had significantly lower levels of DAT and TH mRNA in the VTA than goal-trackers, and of D2R (but not D1R) mRNA in the nucleus accumbens. Given the methodological differences, it is difficult to directly compare our results to the elevated dopamine and DOPAC levels that Tomie (2000) reported in animals that had a tendency to sign-track. Nonetheless, it is apparent that the development of a sign-tracking vs. goal-tracking CR is associated with distinct alterations in dopaminergic systems. Clearly much more work is required to determine the nature and functional significance of these differences. In a potentially related study Uslaner et al. (2007) recently reported that lesions of the subthalamic nucleus greatly enhance sign-tracking towards a cue associated with either food or cocaine reward, implicating this structure as part of the neural system controlling the attribution of incentive salience to reward-related cues.
Stress-response (corticosterone and locomotor response to novelty)
Another physiological correlate of sign-tracking experience is an elevation in plasma corticosterone. Tomie and colleagues have reported that rats receiving paired presentations of a CS-lever and food US (similar to the methods used here) exhibit higher corticosterone levels compared to those in an unpaired or random CS-US group (Tomie et al., 2000; Tomie et al., 2004). Moreover, Tomie (2000) has reported a significant correlation between lever press frequency (or sign-tracking behavior) and corticosterone response (although this was obtained following the 20th autoshaping session). Therefore, in the groups described in Figure 2 we also obtained a measure of plasma corticosterone from tail-nick blood samples before and after the first autoshaping session. Corticosterone was significantly enhanced after the first autoshaping session in all groups, but interestingly, it was elevated to a greater extent in animals that would later be characterized as sign-trackers relative to goal-trackers or the intermediate group (see Figure 4). These data provide, therefore, one physiological response that differentiates the groups even after a single training session. The animals show in Figure 2 were also initially screened for locomotor response to a novel environment and we found no significant correlation between novelty-induced locomotion—an index of novelty-seeking—and subsequent sign-tracking behavior (i.e. probability of approach to the lever; r2 = 0.06, P=0.13), suggesting that these two traits are dissociable (also see Belin et al., 2008).
Studies utilizing selectively bred lines of rats
We have only begun to uncover some of the neurobiological and behavioral correlates of sign-tracking vs. goal-tracking behavior, and how this may be related to the development of compulsive behavioral disorders (Flagel et al., 2008; Tomie et al., 2007). Furthermore, it is not at all clear what factors are responsible for individual differences in responses to reward-related cues, be they behavioral, neurobiological or genetic factors. One approach that may allow us to better explore gene-environment interactions involves ongoing (unpublished) studies using selectively bred lines of rats. It is clear from these ongoing studies that the sign-tracker/goal-tracker trait can be selectively bred, and that it is therefore heritable. However, it is also likely that this trait will be influenced by maternal behavior, developmental events and other environmental factors. Thus, further studies are required to parse out the genetic, psychological and neurobiological mechanisms that underlie various traits that may be important in understanding individual differences in the propensity to develop compulsive behavioral disorders such as addiction (e.g., Belin et al., 2008).
Conclusion: Implications for Addiction
It is the attribution of incentive motivational properties (via Pavlovian learning) that renders reward-predictive cues attractive, desired, and “wanted” (Berridge and Robinson, 2003; Robinson and Berridge, 1993)—and the excessive or pathological attribution of incentive salience to such cues may contribute to the development of compulsive behavioral disorders (Robinson and Berridge, 1993). The idea that reward-related cues can become irresistibly attractive and gain inordinate control over behavior is not a new concept. The phenomenon was described vividly by a gambler in one of the oldest known written texts in the world, the Rig Veda, dating back to 1200 BCE (see O’Flaherty, 1981). In speaking about his dice the gambler stated, “The trembling hazelnut eardrops of the great tree … intoxicate me as they roll on the furrowed board … The dice seem to me like a drink of soma … keeping me awake and excited” … “when the brown dice raise their voice as they are thrown down, I run at once to the rendezvous with them, like a woman to her lover” … “They (the dice) are coated with honey–an irresistible power over the gambler.” Using the present terminology, these dice seem to be attributed with considerable incentive salience.
In conclusion, Pavlovian learning is thought to play a role in maladaptively attracting humans towards drug-associated stimuli and sources of addictive drugs, contributing to the propensity to relapse (Di Chiara, 1998; Everitt and Robbins, 2005; Robinson and Berridge, 1993; Stewart et al., 1984; Tomie, 1996; Tomie et al., 2007). Based on the theoretical accounts and experimental findings presented here, we suggest that individual differences in the tendency to attribute incentive salience to reward-predictive cues may confer vulnerability or resistance to compulsive behavioral disorders, including addiction. Thus, we suggest that the distinction between the sign-tracking and goal-tracking phenotype may provide a means with which to explore individual differences in the tendency to attribute incentive value to reward-predictive cues, and their ability to gain inordinate control over behavior.
Much of our own work presented in this manuscript was supported by grants from the National Institute of Drug Abuse to TER (R37 DA04294) and HA (R01 DA012286) and a program project grant which includes both HA and TER (5P01DA021633-02). The work with the selectively bred animals was also supported by the Office of Naval Research to HA (N00014-02-1-0879). In addition, funding was provided by the Institute for Research on Pathological Gambling and Related Disorders (Cambridge Health Alliance), awarded to SBF. The authors thank Drs. Kent Berridge and Jonathan Morrow for helpful and insightful comments on earlier versions of the manuscript. We would also like to acknowledge the anonymous reviewers for their thoughtful comments which greatly enhanced the quality of this manuscript.
Publisher’s Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
- Anagnostaras SG, Robinson TE. Sensitization to the psychomotor stimulant effects of amphetamine: modulation by associative learning. Behav Neurosci. 1996;110:1397–1414. [PubMed]
- Anagnostaras SG, Schallert T, Robinson TE. Memory processes governing amphetamine-induced psychomotor sensitization. Neuropsychopharmacology. 2002;26:703–715. [PubMed]
- Bardo MT, Bevins RA. Conditioned place preference: what does it add to our preclinical understanding of drug reward? Psychopharmacology (Berl) 2000;153:31–43. [PubMed]
- Belin D, Mar AC, Dalley JW, Robbins TW, Everitt BJ. High impulsivity predicts the switch to compulsive cocaine-taking. Science. 2008;320:1352–1355. [PMC free article] [PubMed]
- Berridge KC. Food reward: brain substrates of wanting and liking. Neurosci Biobehav Rev. 1996;20:1–25. [PubMed]
- Berridge KC. Reward learning: reinforcement, incentives and expectations. In: Medin D, editor. Psychology of learning and motivation. Academic Press; 2001. pp. 223–278.
- Berridge KC, Robinson TE. Parsing reward. Trends Neurosci. 2003;26:507–513. [PubMed]
- Bindra D. How adaptive behavior is produced: a perceptual-motivation alternative to response reinforcement. Behavioral and Brain Sciences. 1978;1:41–91.
- Boakes R. Performance on learning to associate a stimulus with positive reinforcement. In: Davis H, Hurwitz H, editors. Operant-Pavlovian Interactions. Erlbaum; Hillsdale, NJ: 1977. pp. 67–97.
- Boakes RA, Poli M, Lockwood MJ, Goodall G. A study of misbehavior: token reinforcement in the rat. J Exp Anal Behav. 1978;29:115–134. [PMC free article] [PubMed]
- Bolles RC. Reinforcement, expectancy, and learning. Psychol Rev. 1972;79:394–409.
- Boughner RL, Papini MR. Appetitive latent inhibition in rats: now you see it (sign tracking), now you don’t (goal tracking) Learn Behav. 2003;31:387–392. [PubMed]
- Breland K, Breland M. The misbehavior of organisms. Am Psychol. 1961;16:681–683.
- Breland K, Breland M. Animal Behavior. Macmillan; New York: 1966.
- Brown B, Hemmes N, Vaca SCd, Pagano C. Sign and goal tracking during delay and trace autoshaping in pigeons. Animal Learning and Behavior. 1993;21:360–368.
- Brown PL, Jenkins HM. Auto-shaping of the pigeon’s key-peck. J Exp Anal Behav. 1968;11:1–8. [PMC free article] [PubMed]
- Burns M, Domjan M. Sign tracking versus goal tracking in the sexual conditioning of male Japanese quail (Coturnix japonica) J Exp Psychol Anim Behav Process. 1996;22:297–306. [PubMed]
- Burns M, Domjan M. Topography of spatially directed conditioned responding: effects of context and trial duration. J Exp Psychol Anim Behav Process. 2001;27:269–278. [PubMed]
- Buzsaki G. The “where is it?” reflex: autoshaping the orienting response. J Exp Anal Behav. 1982;37:461–484. [PMC free article] [PubMed]
- Cardinal RN, Everitt BJ. Neural and psychological mechanisms underlying appetitive learning: links to drug addiction. Curr Opin Neurobiol. 2004;14:156–162. [PubMed]
- Cardinal RN, Parkinson JA, Hall J, Everitt BJ. Emotion and motivation: the role of the amygdala, ventral striatum, and prefrontal cortex. Neurosci Biobehav Rev. 2002a;26:321–352. [PubMed]
- Cardinal RN, Parkinson JA, Lachenal G, Halkerston KM, Rudarakanchana N, Hall J, Morrison CH, Howes SR, Robbins TW, Everitt BJ. Effects of selective excitotoxic lesions of the nucleus accumbens core, anterior cingulate cortex, and central nucleus of the amygdala on autoshaping performance in rats. Behav Neurosci. 2002b;116:553–567. [PubMed]
- Carr G, Fibiger H, Phillips A. Conditioned place preference as a measure of drug reward. In: Liebman J, Cooper S, editors. The Neuropharmacological Basis of Reward. Oxford University Press; 1989. pp. 264–319.
- Childress AR, Ehrman R, Rohsenow D, Robbins SJ, O’Brien CP. Classically conditioned factors in drug dependence. In: Lowinson J, Millman RP, editors. Comprehensive Textbook of Substance Abuse. Williams and Wilkins; Baltimore: 1993. pp. 56–69.
- Ciccocioppo R, Martin-Fardon R, Weiss F. Stimuli associated with a single cocaine experience elicit long-lasting cocaine-seeking. Nat Neurosci. 2004;7:495–496. [PubMed]
- Cleland GG, Davey GC. Autoshaping in the rat: The effects of localizable visual and auditory signals for food. J Exp Anal Behav. 1983;40:47–56. [PMC free article] [PubMed]
- Corbit LH, Janak PH. Ethanol-associated cues produce general pavlovian-instrumental transfer. Alcohol Clin Exp Res. 2007;31:766–774. [PubMed]
- Costa DS, Boakes RA. Maintenance of responding when reinforcement becomes delayed. Learn Behav. 2007;35:95–105. [PubMed]
- Crombag HS, Shaham Y. Renewal of drug seeking by contextual cues after prolonged extinction in rats. Behav Neurosci. 2002;116:169–173. [PubMed]
- Cunningham CL, Patel P. Rapid induction of Pavlovian approach to an ethanol-paired visual cue in mice. Psychopharmacology (Berl) 2007;192:231–241. [PubMed]
- Cunningham CL, Patel P, Milner L. Spatial location is critical for conditioning place preference with visual but not tactile stimuli. Behav Neurosci. 2006;120:1115–1132. [PubMed]
- Dalley JW, Laane K, Theobald DE, Armstrong HC, Corlett PR, Chudasama Y, Robbins TW. Time-limited modulation of appetitive Pavlovian memory by D1 and NMDA receptors in the nucleus accumbens. Proc Natl Acad Sci U S A. 2005;102:6189–6194. [PMC free article] [PubMed]
- Davey GC, Cleland GG. Topography of signal-centered behavior in the rat: Effects of deprivation state and reinforcer type. J Exp Anal Behav. 1982;38:291–304. [PMC free article] [PubMed]
- Davey GC, Cleland GG, Oakley DA, Jacobs JL. The effect of early feeding experience on signal-directed response topography in the rat. Physiol Behav. 1984;32:11–15. [PubMed]
- Davey GC, Oakley D, Cleland GG. Autoshaping in the rat: Effects of omission on the form of the response. J Exp Anal Behav. 1981;36:75–91. [PMC free article] [PubMed]
- Day JJ, Carelli RM. The nucleus accumbens and Pavlovian reward learning. Neuroscientist. 2007;13:148–159. [PMC free article] [PubMed]
- Day JJ, Wheeler RA, Roitman MF, Carelli RM. Nucleus accumbens neurons encode Pavlovian approach behaviors: evidence from an autoshaping paradigm. Eur J Neurosci. 2006;23:1341–1351. [PubMed]
- DeJong W. Relapse prevention: an emerging technology for promoting long-term drug abstinence. Int J Addict. 1994;29:681–705. [PubMed]
- Di Chiara G. A motivational learning hypothesis of the role of mesolimbic dopamine in compulsive drug use. J Psychopharmacol. 1998;12:54–67. [PubMed]
- Di Ciano P, Everitt BJ. Conditioned reinforcing properties of stimuli paired with self-administered cocaine, heroin or sucrose: implications for the persistence of addictive behaviour. Neuropharmacology. 2004;47 Suppl 1:202–213. [PubMed]
- Dickinson A, Smith J, Mirenowicz J. Dissociation of Pavlovian and instrumental incentive learning under dopamine antagonists. Behav Neurosci. 2000;114:468–483. [PubMed]
- Domjan M, O’Vary D, Greene P. Conditioning of appetitive and consummatory sexual behavior in male Japanese quail. J Exp Anal Behav. 1988;50:505–519. [PMC free article] [PubMed]
- Engberg LA, Hansen G, Welker RL, Thomas DR. Acquisition of key-pecking via autoshaping as a function of prior experience: “learned laziness”? Science. 1972;178:1002–1004. [PubMed]
- Everitt BJ, Robbins TW. Second-order schedules of drug reinforcement in rats and monkeys: measurement of reinforcing efficacy and drug-seeking behaviour. Psychopharmacology (Berl) 2000;153:17–30. [PubMed]
- Everitt BJ, Robbins TW. Neural systems of reinforcement for drug addiction: from actions to habits to compulsion. Nat Neurosci. 2005;8:1481–1489. [PubMed]
- Farwell BJ, Ayres JJB. Stimulus-reinforcer and response-reinforcer relations in the control of conditioned appetitive headpoking (“goal-tracking”) in rats. Learning and Motivation. 1979;10:295–312.
- Flagel SB, Watson SJ, Akil H, Robinson TE. Individual differences in the attribution of incentive salience to a reward-related cue: influence on cocaine sensitization. Behav Brain Res. 2008;186:48–56. [PMC free article] [PubMed]
- Flagel SB, Watson SJ, Robinson TE, Akil H. An animal model of individual differences in “conditionability”: relevance to psychopathology. Neuropsychopharmacology. 2006;31:S262–S263.
- Flagel SB, Watson SJ, Robinson TE, Akil H. Individual differences in the propensity to approach signals vs goals promote different adaptations in the dopamine system of rats. Psychopharmacology (Berl) 2007;191:599–607. [PubMed]
- Gamzu E, Williams DR. Classical conditioning of a complex skeletal response. Science. 1971;171:923–925. [PubMed]
- Hearst E, Jenkins H. Sign-tracking: the stimulus-reinforcer relation and directed action. Monograph of the Psychonomic Society; Austin: 1974.
- Holland PC. Conditioned stimulus as a determinant of the form of the Pavlovian conditioned response. J Exp Psychol Anim Behav Process. 1977;3:77–104. [PubMed]
- Holland PC. CS-US interval as a determinant of the form of Pavlovian appetitive conditioned responses. J Exp Psychol Anim Behav Process. 1980a;6:155–174. [PubMed]
- Holland PC. Influence of visual conditioned stimulus characteristics on the form of Pavlovian appetitive conditioned responding in rats. J Exp Psychol Anim Behav Process. 1980b;6:81–97. [PubMed]
- Holland PC. Occasion setting in Pavlovian conditioning. In: Medin DL, editor. The Psychology of Learning and Motivation. Academic Press; San Diego: 1992. pp. 69–125.
- Jenkins HM, Moore BR. The form of the auto-shaped response with food or water reinforcers. J Exp Anal Behav. 1973;20:163–181. [PMC free article] [PubMed]
- Kearns DN, Gomez-Serrano MA, Weiss SJ, Riley AL. A comparison of Lewis and Fischer rat strains on autoshaping (sign-tracking), discrimination reversal learning and negative auto-maintenance. Behav Brain Res. 2006;169:193–200. [PubMed]
- Kearns DN, Weiss SJ. Sign-tracking (autoshaping) in rats: a comparison of cocaine and food as unconditioned stimuli. Learn Behav. 2004;32:463–476. [PubMed]
- Kemenes G, Benjamin PR. Goal-tracking behavior in the pond snail, Lymnaea stagnalis. Behav Neural Biol. 1989;52:260–270. [PubMed]
- Killeen PR. Complex dynamic processes in sign tracking with an omission contingency (negative automaintenance) J Exp Psychol Anim Behav Process. 2003;29:49–61. [PMC free article] [PubMed]
- Koksal F, Domjan M, Kurt A, Sertel O, Orung S, Bowers R, Kumru G. An animal model of fetishism. Behav Res Ther. 2004;42:1421–1434. [PubMed]
- Krank MD. Pavlovian conditioning with ethanol: sign-tracking (autoshaping), conditioned incentive, and ethanol self-administration. Alcohol Clin Exp Res. 2003;27:1592–1598. [PubMed]
- Krank MD, O’Neill S, Squarey K, Jacob J. Goal- and signal-directed incentive: conditioned approach, seeking, and consumption established with unsweetened alcohol in rats. Psychopharmacology (Berl) 2008;196:397–405. [PubMed]
- Kruzich PJ, Congleton KM, See RE. Conditioned reinstatement of drug-seeking behavior with a discrete compound stimulus classically conditioned with intravenous cocaine. Behav Neurosci. 2001;115:1086–1092. [PubMed]
- Lajoie J, Bindra D. An interpretation of autoshaping and related phenomena in terms of stimulus-incentive contingencies alone. Canadian Journal of Psychology. 1976;30:157–173.
- Locurto C, Terrace HS, Gibbon J. Autoshaping, random control, and omission training in the rat. J Exp Anal Behav. 1976;26:451–462. [PMC free article] [PubMed]
- Locurto CM. Contributions of autoshaping to the partitioning of conditioned bhavior. In: Locurto CM, Terrace HS, Gibbon J, editors. Autoshaping and conditioning theory. Academic Press; New York: 1981. pp. 101–135.
- Lovibond PF. Facilitation of instrumental behavior by a Pavlovian appetitive conditioned stimulus. J Exp Psychol Anim Behav Process. 1983;9:225–247. [PubMed]
- McAlonan GM, Robbins TW, Everitt BJ. Effects of medial dorsal thalamic and ventral pallidal lesions on the acquisition of a conditioned place preference: further evidence for the involvement of the ventral striatopallidal system in reward-related processes. Neuroscience. 1993;52:605–620. [PubMed]
- Moore BR. The role of directed Pavlovian reactions in simple instrumental learning in the pigeon. In: Hinde RA, Stevenson-Hinde J, editors. Constraints on learning: Limitations and predispositions. Academic Press; London: 1973.
- Myerson J, Myerson WA, Parker BK. Automaintenance without stimulus-change reinforcement: Temporal control of key pecks. J Exp Anal Behav. 1979;31:395–403. [PMC free article] [PubMed]
- Newlin DB. A comparison of drug conditioning and craving for alcohol and cocaine. Recent Dev Alcohol. 1992;10:147–164. [PubMed]
- Newlin DB. The self-perceived survival ability and reproductive fitness (SPFit) theory of substance use disorders. Addiction. 2002;97:427–445. [PubMed]
- Nilsson J, Kristiansen TS, Fosseidengen JE, Ferno A, van den Bos R. Sign- and goal-tracking in Atlantic cod (Gadus morhua) Anim Cogn. 2008 In press. [PubMed]
- O’Flaherty WD. The Rig Veda: An Anthology. Penguin Books; New York: 1981. The gamblers lament.
- Pavlov I. Conditioned Reflexes: An Investigation of the Physiological Activity of the Cerebral Cortex. Oxford University Press; London: 1927.
- Pavlov I. The reply of a physiologist to psychologists. Psychol Rev. 1932;39:91–127.
- Peterson GB. Response selection properties of food and brain-stimulation reinforcers in rats. Physiol Behav. 1975;14:681–688. [PubMed]
- Peterson GB, Ackil JE, Frommer GP, Hearst ES. Conditioned approach and contact behavior toward signals for food or brain-stimulation reinforcement. Science. 1972;177:1009–1011. [PubMed]
- Phillips AG, McDonald AC, Wilkie DM. Disruption of autoshaped responding to a signal of brain-stimulation reward by neuroleptic drugs. Pharmacol Biochem Behav. 1981;14:543–548. [PubMed]
- Purdy JE, Roberts AC, Garcia CA. Sign tracking in cuttlefish (Sepia officinalis) J Comp Psychol. 1999;113:443–449. [PubMed]
- Rescorla RA. Pavlovian conditioning. It’s not what you think it is. Am Psychol. 1988;43:151–160. [PubMed]
- Robinson TE, Berridge KC. The neural basis of drug craving: an incentive-sensitization theory of addiction. Brain Res Rev. 1993;18:247–291. [PubMed]
- Sanabria F, Sitomer MT, Killeen PR. Negative automaintenance omission training is effective. J Exp Anal Behav. 2006;86:1–10. [PMC free article] [PubMed]
- Schindler CW, Panlilio LV, Goldberg SR. Second-order schedules of drug self-administration in animals. Psychopharmacology (Berl) 2002;163:327–344. [PubMed]
- Schmajuk NA, Holland PC. Occasion Setting: Associative Learning and Cognition in Animals. American Psychological Association; Washington, D.C.: 1998.
- Schwam E, Gamzu E. Constraints on autoshaping in the squirrel monkey: Stimulus and response factors. Bull Psychonom Soc. 1975;5:369–372.
- Shaham Y, Shalev U, Lu L, De Wit H, Stewart J. The reinstatement model of drug relapse: history, methodology and major findings. Psychopharmacology (Berl) 2003;168:3–20. [PubMed]
- Silva FJ, Silva KM, Pear JJ. Sign- versus goal-tracking: effects of conditioned-stimulus-to-unconditioned-stimulus distance. J Exp Anal Behav. 1992;57:17–31. [PMC free article] [PubMed]
- Skinner BF. The Behavior of Organisms. Appleton-Century-Crofts; New York: 1938.
- Staddon JER, Simmelhag VL. The “superstition” experiment: A reexamination of its implications for the principles of adaptive behavior. Psychol Rev. 1971;78:3–43.
- Stead JD, Clinton S, Neal C, Schneider J, Jama A, Miller S, Vazquez DM, Watson SJ, Akil H. Selective breeding for divergence in novelty-seeking traits: heritability and enrichment in spontaneous anxiety-related behaviors. Behav Genet. 2006;36:697–712. [PubMed]
- Stewart J, de Wit H, Eikelboom R. Role of unconditioned and conditioned drug effects in the self-administration of opiates and stimulants. Psychol Rev. 1984;91:251–268. [PubMed]
- Stiers M, Silberberg A. Lever-contact responses in rats: automaintenance with and without a negative response-reinforcer dependency. J Exp Anal Behav. 1974;22:497–506. [PMC free article] [PubMed]
- Timberlake W, Grant DL. Auto-shaping in rats to the presentation of another rat predicting food. Science. 1975;190:690–692.
- Timberlake W, Lucas GA. The basis of superstitious behavior: chance contingency, stimulus substitution, or appetitive behavior? J Exp Anal Behav. 1985;44:279–299. [PMC free article] [PubMed]
- Toates F. Motivational Systems. Cambridge University Press; Cambridge, UK: 1986.
- Toates F. The interaction of cognitive and stimulus-response processes in the control of behaviour. Neurosci Biobehav Rev. 1998;22:59–83. [PubMed]
- Tomie A. Locating reward cue at response manipulandum (CAM) induces symptoms of drug abuse. Neurosci Biobehav Rev. 1996;20:505–535. [PubMed]
- Tomie A. Autoshaping and drug-taking. In: Mowrer RR, Klein SB, editors. Handbook of Contemporary Learning Theories. Erlbaum; Hillsdale, NJ: 2001. pp. 409–439.
- Tomie A, Aguado AS, Pohorecky LA, Benjamin D. Ethanol induces impulsive-like responding in a delay-of-reward operant choice procedure: impulsivity predicts autoshaping. Psychopharmacology (Berl) 1998;139:376–382. [PubMed]
- Tomie A, Aguado AS, Pohorecky LA, Benjamin D. Individual differences in pavlovian autoshaping of lever pressing in rats predict stress-induced corticosterone release and mesolimbic levels of monoamines. Pharmacol Biochem Behav. 2000;65:509–517. [PubMed]
- Tomie A, Brooks W, Zito B. Sign-tracking: the search for reward. In: Klein S, Mowrer R, editors. Contemporary Learning Theories: Pavlovian Conditioning and the Status of Traditional Learning Theory. Lawrence Erlbaum Associates; Hillsdale, NJ: 1989. pp. 191–223.
- Tomie A, Festa ED, Sparta DR, Pohorecky LA. Lever conditioned stimulus-directed autoshaping induced by saccharin-ethanol unconditioned stimulus solution: effects of ethanol concentration and trial spacing. Alcohol. 2003;30:35–44. [PubMed]
- Tomie A, Grimes KL, Pohorecky LA. Behavioral characteristics and neurobiological substrates shared by Pavlovian sign-tracking and drug abuse. Brain Res Rev. 2007 In press. [PMC free article] [PubMed]
- Tomie A, Tirado AD, Yu L, Pohorecky LA. Pavlovian autoshaping procedures increase plasma corticosterone and levels of norepinephrine and serotonin in prefrontal cortex in rats. Behav Brain Res. 2004;153:97–105. [PubMed]
- Uslaner JM, Acerbo MJ, Jones SA, Robinson TE. The attribution of incentive salience to a stimulus that signals an intravenous injection of cocaine. Behav Brain Res. 2006;169:320–324. [PubMed]
- Uslaner JM, Dell’orco JM, Pevzner A, Robinson TE. The influence of subthalamic nucleus lesions on sign-tracking to stimuli paired with food and drug rewards: facilitation of incentive salience attribution? Neuropsychopharmacology. 2007 In Press. [PubMed]
- Wasserman EA. Pavlovian conditioning with heat reinforcement produces stimulus-directed pecking in chicks. Science. 1973;181:875–877. [PubMed]
- Wessels MG. The effects of reinforcement upon the prepecking behaviors of pigeons in the autoshaping experiment. Journal of the Experimental Analysis of Behavior. 1974;21:125–144. [PMC free article] [PubMed]
- White NM. Addictive drugs as reinforcers: multiple partial actions on memory systems. Addiction. 1996;91:921–949. discussion 951-965. [PubMed]
- White NM, Chai SC, Hamdani S. Learning the morphine conditioned cue preference: cue configuration determines effects of lesions. Pharmacol Biochem Behav. 2005;81:786–796. [PubMed]
- Wilkie DM, McDonald AC. Autoshaping in the rat with electrical stimulation of the brain as the US. Physiol Behav. 1978;21:325–328. [PubMed]
- Williams BA, Dunn R. Preference for conditioned reinforcement. J Exp Anal Behav. 1991;55:37–46. [PMC free article] [PubMed]
- Williams D, Williams H. Automaintenance in the pigeon: Sustained pecking despite contingent non-reinforcement. J Exp Anal Behav. 1969;12:511–520. [PMC free article] [PubMed]
- Wyvell CL, Berridge KC. Intra-accumbens amphetamine increases the conditioned incentive salience of sucrose reward: enhancement of reward “wanting” without enhanced “liking” or response reinforcement. J Neurosci. 2000;20:8122–8130. [PubMed]
- Zamble E, Hadad GM, Mitchell JB, Cutmore TR. Pavlovian conditioning of sexual arousal: first- and second-order effects. J Exp Psychol Anim Behav Process. 1985;11:598–610. [PubMed]
- Zener K. The significance of behavior accompanying conditioned salivary secretion for theories of the conditioned response. Am J Psychol. 1937:384–403.