Back to my home page.
In this trial the model has been trained for 25 "epochs" on a probabilistic learning task. Note that here, R1 "wins" the competition in motor cortex and sends output activation prior to its associated BG Go signal. This is because as a response is increasingly facilitated by the BG in response to an input stimulus, Hebbian principles drive learning directly between the stimulus input and response units in premotor cortex. In effect, the BG/DA system trains the cortical system. This is consistent with observations in both animals and humans that learning related activity occurs in the striatum prior to frontal cortex, and that the BG/DA system is particularly important in the learning of new behaviors, but less important for well ingrained habits. In this particular example, due to the probabilistic nature of the task, negative feedback was delivered.
This trial begins with one response (R3) already active due to it having been selected just before the input stimulus is presented. Note that at the beginning of the trial, R3 motor cortical units are fully active while others are suppressed. But because R3 had not been positively reinforced in response to this new input stimulus, a BG NoGo signal suppresses the initial R3 selection and allows switching to an alternative response (R2). This R2 response is then reinforced with a dopamine burst because it was the correct choice in this particular task context.
In these trials, networks were faced with making a choice in response to two simultaneously presented stimulus cues (two columns of input units), each of which had been separately associated with a different response in the past. In this case, R4 is the correct choice, because it had been associated with the highest probability (80%) of reward, whereas R1 had been associated with 60% reward in response to the other stimulus. This is a high-conflict "win/win" decision, in which the STN is important for preventing premature responding. Noise in motor cortex was increased in this example for demonstration purposes. When the STN is intact it prevents early responding and allows integration of noisy signals; as result the model correctly chooses R4. In contrast, with the STN lesion (inactive STN), the model responds prematurely to R1 as it happens to become more active early in settling of network activity states and is impulsively facilitated.
In this spurious trial, the BG initially facilitates two responses simultaneously, which is not a good thing when having to make choices! However, note that when these two responses are fully excited in premotor cortex, the additional response conflict drives a second STN Global NoGo signal; this leads to excitation of GPi and inhibition of the Thalamus. The lack of bottom-up support for both responses makes it easier for one to dominate and suppress the other (via lateral inhibition that is present in cortex), leading to the selection of just one response. At this point, the conflict in cortex goes down, and the STN Global NoGo signal turns off.
Incorporating Norepinephrine Function into the Model
This model explores the effects of norepinephrine (NE) in modulating cortical response selection processes, as simulated by Aston-Jones, Cohen and colleagues (see Aston-Jones & Cohen, 2005, Annual Review of Neuroscience). Like DA cells in the SNc, firing states of NE-releasing neurons in the locus coeruleus (LC) come in both tonic and phasic modes. In electrophysiological recordings, LC cells release phasic NE bursts during periods of focused attention, infrequent target detection, and good task performance. This phasic NE burst is thought to reflect the outcome of the response selection process and serves to facilitate response execution. In contrast, poor performance is accompanied by a high tonic, but low phasic, state of LC firing. The authors simulated the effects of these LC modes on action selection such that NE modulated the gain of the activation function in cortical response units (Usher et al, 1999). They showed that phasic NE release leads to ``sharper'' cortical representations and a tighter distribution of reaction times, whereas the high tonic state was associated with noisy activity and more RT variability, as observed in their empirical work with monkeys. They further hypothesized that increases in tonic NE during poor performance may be adaptive, in that it may enable the representation of alternate competing cortical actions during exploration of new behaviors.
The below simulations explore how these effects play out within the context of the overall BG/DA action selection circuitry (see Frank, Scheres & Sherman (2007) and Frank, Santamaria, O'Reilly & Willcutt (2007) for simulation results, discussion, and implications for ADHD). We showed that (a) the tonic LC mode leads to increased representation of multiple cortical responses, (b) more reaction time variability, and (c) more erratic trial-to-trial response switching. In the phasic LC mode, tonic LC firing is low but punctate phasic bursts are elicited via top-down excitatory projections from premotor cortex. In this manner stimulus-evoked premotor activity (which arises from prior stimulus-response learning; see above) elicits a phasic LC burst, which in turn reciprocally modulates the gain of premotor units and facilitates the selection and execution of the desired response. These effects turn out to be especially critical in the presence of noisy cortical activity. To explore effects of LC/NE on noisy premotor activity, we delay the stimulus onset so that noisy activity is present in premotor cortex prior to processing of a task-relevant stimulus (as is likely the case in natural environments, but is typically not simulated).
In these trials, noisy activity in premotor units prior to stimulus onset happens to favor the correct cortical response (R1) associated with the particular stimulus. In this "good noise" case, both tonic and phasic LC modes are associated with swift facilitation of the correct response.
In the "bad noise" case, noisy premotor activity prior to stimulus
onset happens to favor R1 units, but R2 is the correct response for
the particular input stimulus. Once the stimulus is presented premotor
R2 units begin to become active. This is because the network had
already been trained sufficiently such that cortical units had
developed strong synaptic strengths directly from the simulus units in
the input layer.