Abstract
Introduction.
The neuromodulator serotonin (5HT) has consistently been implicated in the control of decision-making, although the precise nature of its role, or roles, remains widely debated. Current theories of serotonin function span reward processing, temporal discounting, punishment learning, behavioural flexibility, and behavioural inhibition. It has been difficult to identify any computationally precise mechanism or mode of action.
Methods .
We present data from a pharmacological fMRI study of decision-making in humans, using a task designed to probe precise subcomponents of instrumental learning at both behavioural and neural levels. We manipulated central serotoninergic signaling in a between-subject, double-blind design, using the relatively selective method of acute dietary trypotophan depletion. Subjects selected one of four "bandits" on each trial, with each bandit associated with a nonstationary chance of reward (20 pence) and also a separate chance of punishment (a painful electric shock). Given the choice of a bandit, reward and punishment were delivered independently, allowing their effects to be assessed separately. We characterized choice behavior and associated fMRI signals using reinforcement learning models, and further assessed the serotonergic modulation of behavioral and neural measurements by studying how they covaried, across subjects, with blood tryptophan:LNAA ratios (a marker of the degree of central 5HT depletion).
Results .
Behaviorally, tryptophan status did not significantly influence action learning for either rewards or punishments, or the trade-off between the two (ie, the financial value of the pain implied by the choices). However, serotonin depletion substantially amplified the tendency of subjects to repeat previously chosen actions ("choice stickiness") regardless of the reward or punishment received. This behavior was not explicable as being mediated by any serotonergic effect on uncertainty-driven exploratory chocies. Using concurrent fMRI, we showed that the serotonergic level also predicted the modulation of stickiness-related activity in the medial head of caudate. We further identified both reward- and avoidance-related prediction errors, which though analyzed independently, were found to converge in the same area of caudate, and appeared to constitute a single error signal, with positive BOLD excursions indicating unexpected reward or nonpunishment and negative excursions for unexpected punishment or nonreward. As with the behavior, these reward- and punishment-related signals were not significantly related to serotonin.
Discussion .
Our results support a computational and neurobiological account of serotonin function in which it suppresses the simple, outcome-independent, choice perseveration that characterizes repeated decisions. Such choice stickiness may enhance sampling efficiency in trial-and-error learning (by reducing switching due to local fluctuation in payoffs). Suppressing it would be appropriate in the case of increased optimism about outcomes elsewhere in the environment, and the consequent promotion of exploration may reflect a basic heuristic mechanism of behavioural flexibility that appears to neglect the individual choice uncertainties that would direct an optimal sampling solution. The neural data closely follow the behaviour, and suggest a central role for the head of caudate in integrating distinct components of reinforcement learning and choice.
