Belief inference for hierarchical hidden states in spatial navigation.
Journal
Communications biology
ISSN: 2399-3642
Titre abrégé: Commun Biol
Pays: England
ID NLM: 101719179
Informations de publication
Date de publication:
21 May 2024
21 May 2024
Historique:
received:
22
10
2023
accepted:
10
05
2024
medline:
22
5
2024
pubmed:
22
5
2024
entrez:
21
5
2024
Statut:
epublish
Résumé
Uncertainty abounds in the real world, and in environments with multiple layers of unobservable hidden states, decision-making requires resolving uncertainties based on mutual inference. Focusing on a spatial navigation problem, we develop a Tiger maze task that involved simultaneously inferring the local hidden state and the global hidden state from probabilistically uncertain observation. We adopt a Bayesian computational approach by proposing a hierarchical inference model. Applying this to human task behaviour, alongside functional magnetic resonance brain imaging, allows us to separate the neural correlates associated with reinforcement and reassessment of belief in hidden states. The imaging results also suggest that different layers of uncertainty differentially involve the basal ganglia and dorsomedial prefrontal cortex, and that the regions responsible are organised along the rostral axis of these areas according to the type of inference and the level of abstraction of the hidden state, i.e. higher-order state inference involves more anterior parts.
Identifiants
pubmed: 38773301
doi: 10.1038/s42003-024-06316-0
pii: 10.1038/s42003-024-06316-0
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
614Subventions
Organisme : MEXT | Japan Science and Technology Agency (JST)
ID : JPMJFS2123
Organisme : New Energy and Industrial Technology Development Organization (NEDO)
ID : P20006
Organisme : MEXT | Japan Society for the Promotion of Science (JSPS)
ID : 22H04998
Organisme : MEXT | Japan Society for the Promotion of Science (JSPS)
ID : 23H04676
Organisme : RCUK | Medical Research Council (MRC)
ID : MR/W027593/1
Organisme : Wellcome Trust (Wellcome)
ID : 203139/Z/16/Z
Organisme : Wellcome Trust (Wellcome)
ID : 203139/A/16/Z
Organisme : DH | National Institute for Health Research (NIHR)
ID : NIHR203316
Informations de copyright
© 2024. The Author(s).
Références
Zhao, W. & Chen, W. Hierarchical POMDP planning for object manipulation in clutter. Robot. Auton. Syst. 139, 103736 (2021).
Serrano, S. A., Santiago, E., Martinez-Carranza, J., Morales, E. F. & Sucar, L. E. Knowledge-based hierarchical POMDPs for task planning. J. Intell. Robot. Syst. Theory Appl. 101, 1–23 (2021).
Theocharous, G., Rohanimanesh, K. & Mahadevan, S. Learning hierarchical partially observable Markov decision process models for robot navigation. In Proc. IEEE International Conference on Robotics and Automation Vol. 1 511–516 (IEEE, 2001).
Theocharous, G. & Mahadevan, S. Approximate planning with hierarchical partially observable Markov decision process models for robot navigation. In Proc. IEEE International Conference on Robotics and Automation Vol. 2 1347–1352 (IEEE, 2002).
Qian, K., Ma, X., Dai, X., Fang, F. & Zhou, B. Decision-theoretical navigation of service robots using POMDPs with human-robot co-occurrence prediction. Int. J. Adv. Robot. Syst. 10, 143 (2013).
Gebauer, C., Dengler, N. & Bennewitz, M. Sensor-based navigation using hierarchical reinforcement learning. Lecture Notes Netw. Syst. 577, 546–560 (2023).
doi: 10.1007/978-3-031-22216-0_37
Riesenhuber, M. & Poggio, T. Hierarchical models of object recognition in cortex. Nat. Neurosci. 2, 1019–1025 (1999).
pubmed: 10526343
doi: 10.1038/14819
Cichy, R. M., Khosla, A., Pantazis, D., Torralba, A. & Oliva, A. Comparison of deep neural networks to spatio-temporal cortical dynamics of human visual object recognition reveals hierarchical correspondence. Sci. Rep. 6, 1–13 (2016).
doi: 10.1038/srep27755
Yamins, D. L. K. et al. Performance-optimized hierarchical models predict neural responses in higher visual cortex. Proc. Natl Acad. Sci. USA 111, 8619–8624 (2014).
pubmed: 24812127
pmcid: 4060707
doi: 10.1073/pnas.1403112111
Sarafyazd, M. & Jazayeri, M. Hierarchical reasoning by neural circuits in the frontal cortex. Science 364, eaav8911 (2019).
pubmed: 31097640
doi: 10.1126/science.aav8911
Meyniel, F. & Dehaene, S. Brain networks for confidence weighting and hierarchical inference during probabilistic learning. Proc. Natl Acad. Sci. USA 114, E3859–E3868 (2017).
pubmed: 28439014
pmcid: 5441722
doi: 10.1073/pnas.1615773114
Weilnhammer, V. A., Stuke, H., Sterzer, P. & Schmack, K. The neural correlates of hierarchical predictions for perceptual decisions. J. Neurosci. 38, 5008–5021 (2018).
pubmed: 29712780
pmcid: 6596123
doi: 10.1523/JNEUROSCI.2901-17.2018
Kawato, M., Furukawa, K. & Suzuki, R. A hierarchical neural-network model for control and learning of voluntary movement. Biol. Cybern. 57, 169–185 (1987).
pubmed: 3676355
doi: 10.1007/BF00364149
Ikegami, T. et al. Hierarchical motor adaptations negotiate failures during force field learning. PLoS Comput. Biol. 17, 1–28 (2021).
doi: 10.1371/journal.pcbi.1008481
Stringer, S. M., Rolls, E. T. & Taylor, P. Learning movement sequences with a delayed reward signal in a hierarchical model of motor function. Neural Netw. 20, 172–181 (2007).
pubmed: 16698235
doi: 10.1016/j.neunet.2006.01.016
Ong, W. S., Madlon-Kay, S. & Platt, M. L. Neuronal correlates of strategic cooperation in monkeys. Nat. Neurosci. 24, 116–128 (2021).
pubmed: 33230321
doi: 10.1038/s41593-020-00746-9
Yoshida, W., Seymour, B., Friston, K. J. & Dolan, R. J. Neural mechanisms of belief inference during cooperative games. J. Neurosci. 30, 10744–10751 (2010).
pubmed: 20702705
pmcid: 2967416
doi: 10.1523/JNEUROSCI.5895-09.2010
Diuk, C., Tsai, K., Wallis, J., Botvinick, M. & Niv, Y. Hierarchical learning induces two simultaneous, but separable, prediction errors in human basal ganglia. J. Neurosci. 33, 5797–5805 (2013).
pubmed: 23536092
pmcid: 3865543
doi: 10.1523/JNEUROSCI.5445-12.2013
Yoshida, W. & Ishii, S. Resolution of uncertainty in prefrontal cortex. Neuron 50, 781–789 (2006).
pubmed: 16731515
doi: 10.1016/j.neuron.2006.05.006
Katayama, R., Yoshida, W. & Ishii, S. Confidence modulates the decodability of scene prediction during partially-observable maze exploration in humans. Commun. Biol. 5, 1–14 (2022).
doi: 10.1038/s42003-022-03314-y
Shikauchi, Y. & Ishii, S. Decoding the view expectation during learned maze navigation from human fronto-parietal network. Sci. Rep. 5, 1–13 (2015).
doi: 10.1038/srep17648
Monosov, I. E. Anterior cingulate is a source of valence-specific information about value and uncertainty. Nat. Commun. 8, 1–11 (2017).
doi: 10.1038/s41467-017-00072-y
Behrens, T. E. J., Woolrich, M. W., Walton, M. E. & Rushworth, M. F. S. Learning the value of information in an uncertain world. Nat. Neurosci. 10, 1214–1221 (2007).
pubmed: 17676057
doi: 10.1038/nn1954
Ting, C. C. et al. Neural mechanisms for integrating prior knowledge and likelihood in value-based probabilistic inference. J. Neurosci. 35, 1792–1805 (2015).
pubmed: 25632152
pmcid: 4308614
doi: 10.1523/JNEUROSCI.3161-14.2015
Kolling, N., Behrens, T. E. J., Wittmann, M. K. & Rushworth, M. F. S. Multiple signals in anterior cingulate cortex. Curr. Opin. Neurobiol. 37, 36–43 (2016).
pubmed: 26774693
pmcid: 4863523
doi: 10.1016/j.conb.2015.12.007
Cassandra, A. R., Kaelbling, L. P. & Littman, M. L. Acting optimally in partially observable stochastic domains. Proc. Twelfth Natl Conf. Artif. Intell. 132, 1023–1028 (1995).
Simon, D. A. & Daw, N. D. Neural correlates of forward planning in a spatial decision task in humans. J. Neurosci. 31, 5526–5539 (2011).
pubmed: 21471389
pmcid: 3108440
doi: 10.1523/JNEUROSCI.4647-10.2011
de Cothi, W. et al. Predictive maps in rats and humans for spatial navigation. Curr. Biol. 32, 3676–3689.e5 (2022).
pubmed: 35863351
pmcid: 9616735
doi: 10.1016/j.cub.2022.06.090
Anggraini, D., Glasauer, S. & Wunderlich, K. Neural signatures of reinforcement learning correlate with strategy adoption during spatial navigation. Sci. Rep. 8, 1–14 (2018).
doi: 10.1038/s41598-018-28241-z
Zhu, S., Lakshminarasimhan, K. J., Arfaei, N. & Angelaki, D. E. Eye movements reveal spatiotemporal dynamics of visually-informed planning in navigation. eLife 11, 1–34 (2022).
doi: 10.7554/eLife.73097
Epstein, R. & Kanwisher, N. A cortical representation the local visual environment. Nature 392, 598–601 (1998).
pubmed: 9560155
doi: 10.1038/33402
Yoshida, W., Funakoshi, H. & Ishii, S. Hierarchical rule switching in prefrontal cortex. Neuroimage 50, 314–322 (2009).
pubmed: 20005298
doi: 10.1016/j.neuroimage.2009.12.017
Amemori, K. I. & Graybiel, A. M. Localized microstimulation of primate pregenual cingulate cortex induces negative decision-making. Nat. Neurosci. 15, 776–785 (2012).
pubmed: 22484571
pmcid: 3369110
doi: 10.1038/nn.3088
Zorowitz, S. et al. The neural basis of approach-avoidance conflict: a model based analysis. eNeuro 6, 1–12 (2019).
doi: 10.1523/ENEURO.0115-19.2019
Badre, D., Doll, B. B., Long, N. M. & Frank, M. J. Rostrolateral prefrontal cortex and individual differences in uncertainty-driven exploration. Neuron 73, 595–607 (2012).
pubmed: 22325209
pmcid: 3285405
doi: 10.1016/j.neuron.2011.12.025
Tomov, M. S., Truong, V. Q., Hundia, R. A. & Gershman, S. J. Dissociable neural correlates of uncertainty underlie different exploration strategies. Nat. Commun. 11, 1–12 (2020).
doi: 10.1038/s41467-020-15766-z
Badre, D. & D’Esposito, M. Functional magnetic resonance imaging evidence for a hierarchical organization of the prefrontal cortex. J. Cogn. Neurosci. 19, 2082–2099 (2007).
pubmed: 17892391
doi: 10.1162/jocn.2007.19.12.2082
Braver, T. S. & Bongiolatti, S. R. The role of frontopolar cortex in subgoal processing during working memory. Neuroimage 15, 523–536 (2002).
pubmed: 11848695
doi: 10.1006/nimg.2001.1019
Koechlin, E., Ody, C. & Kouneiher, F. The architecture of cognitive control in the human prefrontal cortex. Science 302, 1181–1185 (2003).
pubmed: 14615530
doi: 10.1126/science.1088545
Badre, D. & Wagner, A. D. Selection, Integration, and conflict monitoring: assessing the nature and generality of prefrontal cognitive control mechanisms. Neuron 41, 473–487 (2004).
pubmed: 14766185
doi: 10.1016/S0896-6273(03)00851-1
Wolfensteller, U. & von Cramon, D. Y. Strategy-effects in prefrontal cortex during learning of higher-order S-R rules. Neuroimage 57, 598–607 (2011).
pubmed: 21554965
doi: 10.1016/j.neuroimage.2011.04.048
Aguirre, G. K., Detre, J. A., Alsop, D. C. & D’Esposito, M. The parahippocampus subserves topographical learning in man. Cereb. Cortex 6, 823–829 (1996).
pubmed: 8922339
doi: 10.1093/cercor/6.6.823
Aguirre, G. K., Zarahn, E. & D’Esposito, M. Neural components of topographical representation. Proc. Natl Acad. Sci. USA 95, 839–846 (1998).
pubmed: 9448249
pmcid: 33806
doi: 10.1073/pnas.95.3.839
Owen, A. M., Milner, B., Petrides, M. & Evans, A. C. A specific role for the right parahippocampal gyrus in the retrieval of object-location: a positron emission tomography study. J. Cogn. Neurosci. 8, 588–602 (1996).
pubmed: 23961986
doi: 10.1162/jocn.1996.8.6.588
Fleming, S. M., Van Der Putten, E. J. & Daw, N. D. Neural mediators of changes of mind about perceptual decisions. Nat. Neurosci. 21, 617–624 (2018).
pubmed: 29531361
pmcid: 5878683
doi: 10.1038/s41593-018-0104-6
Fleck, M. S., Daselaar, S. M., Dobbins, I. G. & Cabeza, R. Role of prefrontal and anterior cingulate regions in decision-making processes shared by memory and nonmemory tasks. Cereb. Cortex 16, 1623–1630 (2006).
pubmed: 16400154
doi: 10.1093/cercor/bhj097
Heereman, J., Walter, H. & Heekeren, H. R. A task-independent neural representation of subjective certainty in visual perception. Front. Hum. Neurosci. 9, 1–12 (2015).
doi: 10.3389/fnhum.2015.00551
Botvinick, M. M., Carter, C. S., Braver, T. S., Barch, D. M. & Cohen, J. D. Conflict monitoring and cognitive control. Psychol. Rev. 108, 624–652 (2001).
pubmed: 11488380
doi: 10.1037/0033-295X.108.3.624
Holroyd, C. B. & Coles, M. G. H. The neural basis of human error processing: Reinforcement learning, dopamine, and the error-related negativity. Psychol. Rev. 109, 679–709 (2002).
pubmed: 12374324
doi: 10.1037/0033-295X.109.4.679
Boldt, A. & Yeung, N. Shared neural markers of decision confidence and error detection. J. Neurosci. 35, 3478–3484 (2015).
pubmed: 25716847
pmcid: 4339357
doi: 10.1523/JNEUROSCI.0797-14.2015
Brown, J. W. & Braver, T. S. Learned predictions of error likelihood in the anterior cingulate cortex. Science 307, 1118–1121 (2005).
pubmed: 15718473
doi: 10.1126/science.1105783
Jessup, R. K., Busemeyer, J. R. & Brown, J. W. Error effects in anterior cingulate cortex reverse when error likelihood is high. J. Neurosci. 30, 3467–3472 (2010).
pubmed: 20203206
pmcid: 2841347
doi: 10.1523/JNEUROSCI.4130-09.2010
Preuschoff, K., Quartz, S. R. & Bossaerts, P. Human insula activation reflects risk prediction errors as well as risk. J. Neurosci. 28, 2745–2752 (2008).
pubmed: 18337404
pmcid: 6670675
doi: 10.1523/JNEUROSCI.4286-07.2008
Loued-Khenissi, L., Pfeuffer, A., Einhäuser, W. & Preuschoff, K. Anterior insula reflects surprise in value-based decision-making and perception. Neuroimage 210, 116549 (2020).
pubmed: 31954844
doi: 10.1016/j.neuroimage.2020.116549
Billeke, P. et al. Human anterior insula encodes performance feedback and relays prediction error to the medial prefrontal cortex. Cereb. Cortex 30, 4011–4025 (2020).
pubmed: 32108230
doi: 10.1093/cercor/bhaa017
Bastin, J. et al. Direct recordings from human anterior insula reveal its leading role within the error-monitoring network. Cereb. Cortex 27, 1545–1557 (2017).
pubmed: 26796212
Su, J., Jia, W. & Wan, X. Task-specific neural representations of generalizable metacognitive control signals in the human dorsal anterior cingulate cortex. J. Neurosci. 42, 1275–1291 (2022).
pubmed: 34907025
pmcid: 8883847
doi: 10.1523/JNEUROSCI.1283-21.2021
Pereira, M. et al. Disentangling the origins of confidence in speeded perceptual judgments through multimodal imaging. Proc. Natl Acad. Sci. USA 117, 8382–8390 (2020).
pubmed: 32238562
pmcid: 7165419
doi: 10.1073/pnas.1918335117
Fleming, S. M., Huijgen, J. & Dolan, R. J. Prefrontal contributions to metacognition in perceptual decision making. J. Neurosci. 32, 6117–6125 (2012).
pubmed: 22553018
pmcid: 3359781
doi: 10.1523/JNEUROSCI.6489-11.2012
Glaser, J. I., Perich, M. G., Ramkumar, P., Miller, L. E. & Kording, K. P. Population coding of conditional probability distributions in dorsal premotor cortex. Nat. Commun. 9, 1788 (2018).
pubmed: 29725023
pmcid: 5934453
doi: 10.1038/s41467-018-04062-6
Chan, S. C. Y., Niv, Y. & Norman, K. A. A probability distribution over latent causes, in the orbitofrontal cortex. J. Neurosci. 36, 7817–7828 (2016).
pubmed: 27466328
pmcid: 4961772
doi: 10.1523/JNEUROSCI.0659-16.2016
Vilares, I., Howard, J. D., Fernandes, H. L., Gottfried, J. A. & Kording, K. P. Differential representations of prior and likelihood uncertainty in the human brain. Curr. Biol. 22, 1641–1648 (2012).
pubmed: 22840519
pmcid: 3461114
doi: 10.1016/j.cub.2012.07.010
Plaza, P., Cuevas, I., Grandin, C., De Volder, A. G. & Renier, L. Looking into task-specific activation using a prosthesis substituting vision with audition. ISRN Rehabil. 2012, 1–15 (2012).
doi: 10.5402/2012/490950
Chen, Y. et al. Allocentric versus egocentric representation of remembered reach targets in human cortex. J. Neurosci. 34, 12515–12526 (2014).
pubmed: 25209289
pmcid: 6615499
doi: 10.1523/JNEUROSCI.1445-14.2014
Lester, B. D. & Dassonville, P. The role of the right superior parietal lobule in processing visual context for the establishment of the egocentric reference frame. J. Cogn. Neurosci. 26, 2201–2209 (2014).
pubmed: 24702459
doi: 10.1162/jocn_a_00636
Neggers, S. F. W., Van der Lubbe, R. H. J., Ramsey, N. F. & Postma, A. Interactions between ego- and allocentric neuronal representations of space. Neuroimage 31, 320–331 (2006).
pubmed: 16473025
doi: 10.1016/j.neuroimage.2005.12.028
Rodriguez, P. F. Neural decoding of goal locations in spatial navigation in humans with fMRI. Hum. Brain Mapp. 31, 391–397 (2010).
pubmed: 19722170
doi: 10.1002/hbm.20873
Sherrill, K. R. et al. Functional connections between optic flow areas and navigationally responsive brain regions during goal-directed navigation. Neuroimage 118, 386–396 (2015).
pubmed: 26054874
doi: 10.1016/j.neuroimage.2015.06.009
Spiers, H. J. & Maguire, E. A. A navigational guidance system in the human brain. Hippocampus 17, 618–626 (2007).
pubmed: 17492693
pmcid: 2570439
doi: 10.1002/hipo.20298
Schwartenbeck, P., FitzGerald, T. H. B. & Dolan, R. Neural signals encoding shifts in beliefs. Neuroimage 125, 578–586 (2016).
pubmed: 26520774
doi: 10.1016/j.neuroimage.2015.10.067
O’Reilly, J. X. et al. Dissociable effects of surprise and model update in parietal and anterior cingulate cortex. Proc. Natl Acad. Sci. USA 110, E3660–E3669 (2013).
pubmed: 23986499
pmcid: 3780876
doi: 10.1073/pnas.1305373110
Boorman, E. D., Rajendran, V. G., O’Reilly, J. X. & Behrens, T. E. Two anatomically and computationally distinct learning signals predict changes to stimulus-outcome associations in Hippocampus. Neuron 89, 1343–1354 (2016).
pubmed: 26948895
pmcid: 4819449
doi: 10.1016/j.neuron.2016.02.014
Hunt, L. T. et al. Triple dissociation of attention and decision computations across prefrontal cortex. Nat. Neurosci. 21, 1471–1481 (2018).
pubmed: 30258238
pmcid: 6331040
doi: 10.1038/s41593-018-0239-5
Starkweather, C. K., Gershman, S. J. & Uchida, N. The medial prefrontal cortex shapes dopamine reward prediction errors under state uncertainty. Neuron 98, 616–629.e6 (2018).
pubmed: 29656872
pmcid: 5934341
doi: 10.1016/j.neuron.2018.03.036
Shenhav, A., Straccia, M. A., Musslick, S., Cohen, J. D. & Botvinick, M. M. Dissociable neural mechanisms track evidence accumulation for selection of attention versus action. Nat. Commun. 9, 2485 (2018).
pubmed: 29950596
pmcid: 6021379
doi: 10.1038/s41467-018-04841-1
Taren, A. A., Venkatraman, V. & Huettel, S. A. A parallel functional topography between medial and lateral prefrontal cortex: evidence and implications for cognitive control. J. Neurosci. 31, 5026 (2011).
pubmed: 21451040
pmcid: 3096488
doi: 10.1523/JNEUROSCI.5762-10.2011
Venkatraman, V., Rosati, A. G., Taren, A. A. & Huettel, S. A. Resolving response, decision, and strategic control: evidence for a functional topography in dorsomedial prefrontal cortex. J. Neurosci. 29, 13158 (2009).
pubmed: 19846703
pmcid: 2801415
doi: 10.1523/JNEUROSCI.2708-09.2009
Zarr, N. & Brown, J. W. Hierarchical error representation in medial prefrontal cortex. Neuroimage 124, 238–247 (2016).
pubmed: 26343320
doi: 10.1016/j.neuroimage.2015.08.063
Holroyd, C. B. & McClure, S. M. Hierarchical control over effortful behavior by rodent medial frontal cortex: a computational model. Psychol. Rev. 122, 54–83 (2015).
pubmed: 25437491
doi: 10.1037/a0038339
Shenhav, A., Botvinick, M. M. & Cohen, J. D. The expected value of control: an integrative theory of anterior cingulate cortex function. Neuron 79, 217–240 (2013).
pubmed: 23889930
pmcid: 3767969
doi: 10.1016/j.neuron.2013.07.007
Vassena, E., Holroyd, C. B. & Alexander, W. H. Computational models of anterior cingulate cortex: at the crossroads between prediction and effort. Front. Neurosci. 11, 316 (2017).
pubmed: 28634438
pmcid: 5459890
doi: 10.3389/fnins.2017.00316
de Wit, S. et al. Corticostriatal connectivity underlies individual differences in the balance between habitual and goal-directed action control. J. Neurosci. 32, 12066 (2012).
pubmed: 22933790
pmcid: 6621537
doi: 10.1523/JNEUROSCI.1088-12.2012
Tricomi, E., Balleine, B. W. & O’Doherty, J. P. A specific role for posterior dorsolateral striatum in human habit learning. Eur. J. Neurosci. 29, 2225–2232 (2009).
pubmed: 19490086
pmcid: 2758609
doi: 10.1111/j.1460-9568.2009.06796.x
Horga, G. et al. Changes in corticostriatal connectivity during reinforcement learning in humans. Hum. Brain Mapp. 36, 793–803 (2015).
pubmed: 25393839
doi: 10.1002/hbm.22665
Wan Lee, S., Shimojo, S. & O’Doherty, J. P. Neural computations underlying arbitration between model-based and model-free learning. Neuron 81, 687 (2014).
doi: 10.1016/j.neuron.2013.11.028
Duan, L. Y. et al. Controlling one’s world: identification of sub-regions of primate PFC underlying goal-directed behavior. Neuron 109, 2485 (2021).
pubmed: 34171290
pmcid: 8346232
doi: 10.1016/j.neuron.2021.06.003
Fujimoto, A. et al. Signaling incentive and drive in the primate ventral pallidum for motivational control of goal-directed action. J. Neurosci. 39, 1793–1804 (2019).
pubmed: 30626695
pmcid: 6407294
doi: 10.1523/JNEUROSCI.2399-18.2018
Turner, K. M., Svegborn, A., Langguth, M., McKenzie, C. & Robbins, T. W. Opposing roles of the dorsolateral and dorsomedial striatum in the acquisition of skilled action sequencing in Rats. J. Neurosci. 42, 2039–2051 (2022).
pubmed: 35086903
pmcid: 8916752
doi: 10.1523/JNEUROSCI.1907-21.2022
Gremel, C. M. & Costa, R. M. Orbitofrontal and striatal circuits dynamically encode the shift between goal-directed and habitual actions. Nat. Commun. 4, 2264 (2013).
pubmed: 23921250
doi: 10.1038/ncomms3264
Peirce, J. et al. PsychoPy2: experiments in behavior made easy. Behav. Res. Methods 51, 195–203 (2019).
pubmed: 30734206
pmcid: 6420413
doi: 10.3758/s13428-018-01193-y
R. Core Team. R: A Language and Environment for Statistical Computing (R Foundation for Statistical Computing: Vienna, Austria, 2017).