Imprecise neural computations as a source of adaptive behaviour in volatile environments.
Journal
Nature human behaviour
ISSN: 2397-3374
Titre abrégé: Nat Hum Behav
Pays: England
ID NLM: 101697750
Informations de publication
Date de publication:
01 2021
01 2021
Historique:
received:
31
10
2019
accepted:
18
09
2020
pubmed:
11
11
2020
medline:
5
3
2021
entrez:
10
11
2020
Statut:
ppublish
Résumé
In everyday life, humans face environments that feature uncertain and volatile or changing situations. Efficient adaptive behaviour must take into account uncertainty and volatility. Previous models of adaptive behaviour involve inferences about volatility that rely on complex and often intractable computations. Because such computations are presumably implausible biologically, it is unclear how humans develop efficient adaptive behaviours in such environments. Here, we demonstrate a counterintuitive result: simple, low-level inferences confined to uncertainty can produce near-optimal adaptive behaviour, regardless of the environmental volatility, assuming imprecisions in computation that conform to the psychophysical Weber law. We further show empirically that this Weber-imprecision model explains human behaviour in volatile environments better than optimal adaptive models that rely on high-level inferences about volatility, even when considering biologically plausible approximations of such models, as well as non-inferential models like adaptive reinforcement learning.
Identifiants
pubmed: 33168951
doi: 10.1038/s41562-020-00971-z
pii: 10.1038/s41562-020-00971-z
doi:
Types de publication
Journal Article
Research Support, Non-U.S. Gov't
Langues
eng
Sous-ensembles de citation
IM
Pagination
99-112Subventions
Organisme : EC | EC Seventh Framework Programm | FP7 Ideas: European Research Council (FP7-IDEAS-ERC - Specific Programme: "Ideas" Implementing the Seventh Framework Programme of the European Community for Research, Technological Development and Demonstration Activities (2007 to 2013))
ID : AdG #250106
Commentaires et corrections
Type : CommentIn
Références
Behrens, T. E., Woolrich, M. W., Walton, M. E. & Rushworth, M. F. Learning the value of information in an uncertain world. Nat. Neurosci. 10, 1214–1221 (2007).
doi: 10.1038/nn1954
Boorman, E. D., Behrens, T. E., Woolrich, M. W. & Rushworth, M. F. How green is the grass on the other side? Frontopolar cortex and the evidence in favor of alternative courses of action. Neuron 62, 733–743 (2009).
doi: 10.1016/j.neuron.2009.05.014
Gershman, S. J., Blei, D. M. & Niv, Y. Context learning, and extinction. Psychological Rev. 117, 1997–1209 (2010).
doi: 10.1037/a0017808
Tenenbaum, J. B., Kemp, C., Griffiths, T. L. & Goodman, N. D. How to grow a mind: statistics, structure, and abstraction. Science 331, 1279–1285 (2011).
doi: 10.1126/science.1192788
Collins, A. G. & Koechlin, E. Reasoning, learning, and creativity: frontal lobe function and human decision-making. PLoS Biol. 10, e1001293 (2012).
doi: 10.1371/journal.pbio.1001293
Collins, A. G. & Frank, M. J. Cognitive control over learning: creating, clustering, and generalizing task-set structure. Psychol. Rev. 120, 190–229 (2013).
doi: 10.1037/a0030852
Donoso, M., Collins, A. G. & Koechlin, E. Foundations of human reasoning in the prefrontal cortex. Science 344, 1481–1486 (2014).
doi: 10.1126/science.1252254
Kolossa, A., Kopp, B. & Fingscheidt, T. A computational analysis of the neural bases of Bayesian inference. Neuroimage 106, 222–237 (2015).
doi: 10.1016/j.neuroimage.2014.11.007
Schuck, N. W., Cai, M. B., Wilson, R. C. & Niv, Y. Human orbitofrontal cortex represents a cognitive map of state space. Neuron 91, 1402–1412 (2016).
doi: 10.1016/j.neuron.2016.08.019
Rouault, M., Drugowitsch, J. & Koechlin, E. Prefrontal mechanisms combining rewards and beliefs in human decision-making. Nat. Commun. 10, 301 (2019).
doi: 10.1038/s41467-018-08121-w
Nassar, M. R., Wilson, R. C., Heasly, B. & Gold, J. I. An approximately Bayesian delta-rule model explains the dynamics of belief updating in a changing environment. J. Neurosci. 30, 12366–12378 (2010).
doi: 10.1523/JNEUROSCI.0822-10.2010
Payzan-LeNestour, E. & Bossaerts, P. Risk, unexpected uncertainty, and estimation uncertainty: Bayesian learning in unstable settings. PLoS Comput. Biol. 7, e1001048 (2011).
doi: 10.1371/journal.pcbi.1001048
Wilson, R. C., Nassar, M. R. & Gold, J. I. A mixture of delta-rules approximation to bayesian inference in change-point problems. PLoS Comput. Biol. 9, e1003150 (2013).
doi: 10.1371/journal.pcbi.1003150
McGuire, J. T., Nassar, M. R., Gold, J. I. & Kable, J. W. Functionally dissociable influences on learning rate in a dynamic environment. Neuron 84, 870–881 (2014).
doi: 10.1016/j.neuron.2014.10.013
Bossaerts, P., Yadav, N. & Murawski, C. Uncertainty and computational complexity. Philos. Trans. R. Soc. Lond. B Biol. Sci. 374, 20180138 (2019).
doi: 10.1098/rstb.2018.0138
Drugowitsch, J., Wyart, V., Devauchelle, A. D. & Koechlin, E. Computational precision of mental inference as critical source of human choice suboptimality. Neuron 92, 1398–1411 (2016).
doi: 10.1016/j.neuron.2016.11.005
Fechner G. T. Elemente der Psychophysik (Breitkopf and Härtel, 1860).
Treisman, M. Noise and Weber’s law: the discrimination of brightness and other dimensions. Psychol. Rev. 71, 314–330 (1964).
doi: 10.1037/h0042445
Deco, G., Scarano, L. & Soto-Faraco, S. Weber’s law in decision making: integrating behavioral data in humans with a neurophysiological model. J. Neurosci. 27, 11192–11200 (2007).
doi: 10.1523/JNEUROSCI.1072-07.2007
Wyart, V. & Koechlin, E. Choice variability and suboptimality in uncertain environments. Curr. Opin. Behav. Sci. 11, 109–115 (2016).
doi: 10.1016/j.cobeha.2016.07.003
Faraji, M., Preuschoff, K. & Gerstner, W. Balancing new against old information: the role of puzzlement surprise in learning. Neural Comput. 30, 34–83 (2018).
doi: 10.1162/neco_a_01025
Chopin, N., Jacob, P. E. & Papaspiliopoulos, O. SMC2: an efficient algorithm for sequential analysis of state space models. J. R. Stat. Soc. Series B Stat. Methodol. 75, 397–426 (2013).
doi: 10.1111/j.1467-9868.2012.01046.x
Doucet, A., Godsill, S. & Andrieu, C. On sequential Monte Carlo sampling methods for Bayesian filtering. Statist. Comput. 10, 197–208 (2000).
doi: 10.1023/A:1008935410038
Andrieu, C., Doucet, A. & Holenstein, R. Particle Markov chain Monte Carlo methods. J. R. Stat. Soc. Series B Stat. Methodol. 72, 269–342 (2010).
doi: 10.1111/j.1467-9868.2009.00736.x
Chopin, N. A sequential particle filter for static models. Biometrika 89, 539–552 (2002).
doi: 10.1093/biomet/89.3.539
Shi, L. & Griffiths, T. L. Neural implementation of hierarchical Bayesian inference by importance sampling. Adv. Neural Inf. Process. Syst. 22, 1669–1677 (2009).
Huang, Y. & Rao, R. P. Neurons as Monte Carlo samplers: Bayesian inference and learning in spiking networks. Adv. Neural Inf. Process. Syst. 27, 1943–1951 (2014).
Legenstein, R. & Maass, W. Ensembles of spiking neurons with noise support optimal probabilistic inference in a dynamically changing environment. PLoS Comput. Biol. 10, e1003859 (2014).
doi: 10.1371/journal.pcbi.1003859
Scott, S. L. Bayesian methods for hidden Markov models. J. Am. Stat. Assoc. 97, 337–351 (2002).
doi: 10.1198/016214502753479464
Pearce, J. M. & Hall, G. A model for Pavlovian learning: variations in the effectiveness of conditioned but not of unconditioned stimuli. Psychol. Rev. 87, 532–552 (1980).
doi: 10.1037/0033-295X.87.6.532
Roesch, M., Esber, G. R., Li, J., Daw, N. & Schoenbaum, G. Surprise! Neural correlates of Pearce–Hall and Rescorla–Wagner coexist within the brain. Eur. J. Neurosci. 35, 1190–1200 (2012).
doi: 10.1111/j.1460-9568.2011.07986.x
Daw, N. D., O’Doherty, J. P., Dayan, P., Seymour, B. & Dolan, R. J. Cortical substrates for exploratory decisions in humans. Nature 441, 876–879 (2006).
doi: 10.1038/nature04766
Rigoux, L., Stephan, K. E., Friston, K. J. & Daunizeau, J. Bayesian model selection for group studies - revisited. Neuroimage 84, 971–985 (2014).
doi: 10.1016/j.neuroimage.2013.08.065
Stephan, K. E., Penny, W. D., Daunizeau, J., Moran, R. J. & Friston, K. J. Bayesian model selection for group studies. Neuroimage 46, 1004–1017 (2009).
doi: 10.1016/j.neuroimage.2009.03.025
Palminteri, S., Wyart, V. & Koechlin, E. The importance of falsification in computational cognitive modeling. Trends Cogn. Sci. 21, 425–433 (2017).
doi: 10.1016/j.tics.2017.03.011
Payzan-LeNestour, E. Bayesian Learning in Unstable Settings: Experimental Evidence Based on the Bandit Problem Research Paper No. 10-28 (Swiss Finance Inst., 2010).
Summerfield, C., Behrens, T. E. & Koechlin, E. Perceptual classification in a rapidly changing environment. Neuron 71, 725–736 (2011).
doi: 10.1016/j.neuron.2011.06.022
Knight, F. H. Risk, Uncertainty and Profit (Univ. Chicago Press, 1921).
Keynes, J. M. A Treatise on Probability (Macmillan, 1921).
Bogacz, R., Brown, E., Moehlis, J., Holmes, P. & Cohen, J. D. The physics of optimal decision making: a formal analysis of models of performance in two-alternative forced-choice tasks. Psychol. Rev. 113, 700–765 (2006).
doi: 10.1037/0033-295X.113.4.700
Wang, X.-J. Neural dynamics and circuit mechnaisms of decision-making. Curr. Opin. Neurobiol. 22, 1039–1046 (2012).
doi: 10.1016/j.conb.2012.08.006
Payzan-LeNestour, E., Dunne, S., Bossaerts, P. & O’Doherty, J. P. The neural representation of unexpected uncertainty during value-based decision making. Neuron 79, 191–201 (2013).
doi: 10.1016/j.neuron.2013.04.037
Lebreton, M., Abitbol, R., Daunizeau, J. & Pessiglione, M. Automatic integration of confidence in the brain valuation signal. Nat. Neurosci. 18, 1159–1167 (2015).
doi: 10.1038/nn.4064
Beaumont, M. A. Estimation of population growth or decline in genetically monitored populations. Genetics 164, 1139–1160 (2003).
pubmed: 12871921
pmcid: 1462617
Niederreiter, H. Random Number Generation and Quasi-Monte Carlo Methods (Society for Industrial and Applied Mathematics, 1992).
Snoek, J., Larochelle, H. & Adams, R. P. Practical Bayesian optimization of machine learning algorithms. Preprint at arXiv http://arxiv.org/abs/1206.2944 (2012).