Transition from model-free to structure-informed decision making in dynamic environments

Authors

Yasueda, M., Taira, M., Akam, T., Walton, M. E., Doya, K.

Abstract

Reinforcement learning theory formulates distinct decision-making strategies, including reactive model-free and deliberative model-based strategies. This study investigates how mice adjust their reinforcement learning strategies while learning decision-making in dynamic environments. Unlike previous studies that focused on behaviors after extensive training periods, we analyzed changes in learning strategies in the course of training of a two-step decision-making task with probabilistic state transition and fluctuating reward probabilities. Our statistical behavioral analysis showed that the stay-probability following common and rare transitions diverged with training, a signature of strategies that utilize knowledge of task structure. We fit various reinforcement learning strategies to behavioral data and found that structure-informed strategies became increasingly dominant in their behaviors during training. Whereas previous studies emphasized transition from goal-directed to habitual strategies after extensive training, which were often associated with model-based and model-free strategies, respectively, our results newly demonstrate a shift from model-free to structure-informed strategies in early training in mice.

Preprint server: bioRxiv
The authors list and abstract were imported from bioRxiv on 19 Aug 2026.

Stats

1-terrible, 9-excellent. How would you rate this preprint? Sign in in to submit your rating.

Post a comment

You need to be signed in to post comments. You can sign in here.

Comments

There are no comments yet.