Header

UZH-Logo

Maintenance Infos

Neurostimulation reveals context-dependent arbitration between model-based and model-free reinforcement learning


Weissengruber, Sebastian; Lee, Sang Wan; O’Doherty, John P; Ruff, Christian C (2019). Neurostimulation reveals context-dependent arbitration between model-based and model-free reinforcement learning. Cerebral Cortex, 29(11):4850-4862.

Abstract

While it is established that humans use model-based (MB) and model-free (MF) reinforcement learning in a complementary fashion, much less is known about how the brain determines which of these systems should control behavior at any given moment. Here we provide causal evidence for a neural mechanism that acts as a context-dependent arbitrator between both systems. We applied excitatory and inhibitory transcranial direct current stimulation over a region of the left ventrolateral prefrontal cortex previously found to encode the reliability of both learning systems. The opposing neural interventions resulted in a bidirectional shift of control between MB and MF learning. Stimulation also affected the sensitivity of the arbitration mechanism itself, as it changed how often subjects switched between the dominant system over time. Both of these effects depended on varying task contexts that either favored MB or MF control, indicating that this arbitration mechanism is not context-invariant but flexibly incorporates information about current environmental demands.

Abstract

While it is established that humans use model-based (MB) and model-free (MF) reinforcement learning in a complementary fashion, much less is known about how the brain determines which of these systems should control behavior at any given moment. Here we provide causal evidence for a neural mechanism that acts as a context-dependent arbitrator between both systems. We applied excitatory and inhibitory transcranial direct current stimulation over a region of the left ventrolateral prefrontal cortex previously found to encode the reliability of both learning systems. The opposing neural interventions resulted in a bidirectional shift of control between MB and MF learning. Stimulation also affected the sensitivity of the arbitration mechanism itself, as it changed how often subjects switched between the dominant system over time. Both of these effects depended on varying task contexts that either favored MB or MF control, indicating that this arbitration mechanism is not context-invariant but flexibly incorporates information about current environmental demands.

Statistics

Citations

Altmetrics

Downloads

2 downloads since deposited on 30 Jul 2019
2 downloads since 12 months
Detailed statistics

Additional indexing

Item Type:Journal Article, refereed, original work
Communities & Collections:03 Faculty of Economics > Department of Economics
Dewey Decimal Classification:330 Economics
Uncontrolled Keywords:Cognitive neuroscience, cellular and molecular neuroscience, goal-directed, habitual, reinforcement learning, tDCS, ventrolateral PFC
Language:English
Date:17 December 2019
Deposited On:30 Jul 2019 09:51
Last Modified:19 Mar 2020 01:00
Publisher:Oxford University Press
ISSN:1047-3211
Additional Information:This is a pre-copy-editing, author-produced PDF of an article accepted for publication in Cerebral Cortex following peer review. The definitive publisher-authenticated version of "Neurostimulation reveals context-dependent arbitration between model-based and model-free reinforcement learning" is available online at https://doi.org/10.1093/cercor/bhz019.
OA Status:Green
Free access at:Publisher DOI. An embargo period may apply.
Publisher DOI:https://doi.org/10.1093/cercor/bhz019
Project Information:
  • : FunderSNSF
  • : Grant IDCRSII3_141965
  • : Project TitleNeuroeconomics of value-based decision making

Download

Green Open Access

Download PDF  'Neurostimulation reveals context-dependent arbitration between model-based and model-free reinforcement learning'.
Preview
Content: Accepted Version
Filetype: PDF
Size: 1MB
View at publisher