Advanced search
1 file | 2.47 MB Add to list

Learning the value of information and reward over time when solving exploration-exploitation problems

Author
Organization
Abstract
To flexibly adapt to the demands of their environment, animals are constantly exposed to the conflict resulting from having to choose between predictably rewarding familiar options (exploitation) and risky novel options, the value of which essentially consists of obtaining new information about the space of possible rewards (exploration). Despite extensive research, the mechanisms that subtend the manner in which animals solve this exploitation-exploration dilemma are still poorly understood. Here, we investigate human decision-making in a gambling task in which the informational value of each trial and the reward potential were separately manipulated. To better characterize the mechanisms that underlined the observed behavioural choices, we introduce a computational model that augments the standard reward-based reinforcement learning formulation by associating a value to information. We find that both reward and information gained during learning influence the balance between exploitation and exploration, and that this influence was dependent on the reward context. Our results shed light on the mechanisms that underpin decision-making under uncertainty, and suggest new approaches for investigating the exploration-exploitation dilemma throughout the animal kingdom.
Keywords
DECISION-MAKING, UNCERTAINTY, HUMANS, BRAIN, MODEL, RISK

Downloads

  • Dezza2017SciRep-1.pdf
    • full text
    • |
    • open access
    • |
    • PDF
    • |
    • 2.47 MB

Citation

Please use this url to cite or link to this publication:

MLA
Cogliati Dezza, Irene, et al. “Learning the Value of Information and Reward over Time When Solving Exploration-Exploitation Problems.” SCIENTIFIC REPORTS, vol. 7, Nature Publishing Group, 2017, doi:10.1038/s41598-017-17237-w.
APA
Cogliati Dezza, I., Yu, A. J., Cleeremans, A., & Alexander, W. (2017). Learning the value of information and reward over time when solving exploration-exploitation problems. SCIENTIFIC REPORTS, 7. https://doi.org/10.1038/s41598-017-17237-w
Chicago author-date
Cogliati Dezza, Irene, Angela J. Yu, Axel Cleeremans, and William Alexander. 2017. “Learning the Value of Information and Reward over Time When Solving Exploration-Exploitation Problems.” SCIENTIFIC REPORTS 7. https://doi.org/10.1038/s41598-017-17237-w.
Chicago author-date (all authors)
Cogliati Dezza, Irene, Angela J. Yu, Axel Cleeremans, and William Alexander. 2017. “Learning the Value of Information and Reward over Time When Solving Exploration-Exploitation Problems.” SCIENTIFIC REPORTS 7. doi:10.1038/s41598-017-17237-w.
Vancouver
1.
Cogliati Dezza I, Yu AJ, Cleeremans A, Alexander W. Learning the value of information and reward over time when solving exploration-exploitation problems. SCIENTIFIC REPORTS. 2017;7.
IEEE
[1]
I. Cogliati Dezza, A. J. Yu, A. Cleeremans, and W. Alexander, “Learning the value of information and reward over time when solving exploration-exploitation problems,” SCIENTIFIC REPORTS, vol. 7, 2017.
@article{8552704,
  abstract     = {{To flexibly adapt to the demands of their environment, animals are constantly exposed to the conflict resulting from having to choose between predictably rewarding familiar options (exploitation) and risky novel options, the value of which essentially consists of obtaining new information about the space of possible rewards (exploration). Despite extensive research, the mechanisms that subtend the manner in which animals solve this exploitation-exploration dilemma are still poorly understood. Here, we investigate human decision-making in a gambling task in which the informational value of each trial and the reward potential were separately manipulated. To better characterize the mechanisms that underlined the observed behavioural choices, we introduce a computational model that augments the standard reward-based reinforcement learning formulation by associating a value to information. We find that both reward and information gained during learning influence the balance between exploitation and exploration, and that this influence was dependent on the reward context. Our results shed light on the mechanisms that underpin decision-making under uncertainty, and suggest new approaches for investigating the exploration-exploitation dilemma throughout the animal kingdom.}},
  articleno    = {{16919}},
  author       = {{Cogliati Dezza, Irene and Yu, Angela J. and Cleeremans, Axel and Alexander, William}},
  issn         = {{2045-2322}},
  journal      = {{SCIENTIFIC REPORTS}},
  keywords     = {{DECISION-MAKING,UNCERTAINTY,HUMANS,BRAIN,MODEL,RISK}},
  language     = {{eng}},
  pages        = {{13}},
  publisher    = {{Nature Publishing Group}},
  title        = {{Learning the value of information and reward over time when solving exploration-exploitation problems}},
  url          = {{http://doi.org/10.1038/s41598-017-17237-w}},
  volume       = {{7}},
  year         = {{2017}},
}

Altmetric
View in Altmetric
Web of Science
Times cited: