The impact of learning on perceptual decisions and its implication for speed-accuracy tradeoffs.
Journal
Nature communications
ISSN: 2041-1723
Titre abrégé: Nat Commun
Pays: England
ID NLM: 101528555
Informations de publication
Date de publication:
02 06 2020
02 06 2020
Historique:
received:
09
03
2019
accepted:
01
04
2020
entrez:
4
6
2020
pubmed:
4
6
2020
medline:
22
8
2020
Statut:
epublish
Résumé
In standard models of perceptual decision-making, noisy sensory evidence is considered to be the primary source of choice errors and the accumulation of evidence needed to overcome this noise gives rise to speed-accuracy tradeoffs. Here, we investigated how the history of recent choices and their outcomes interact with these processes using a combination of theory and experiment. We found that the speed and accuracy of performance of rats on olfactory decision tasks could be best explained by a Bayesian model that combines reinforcement-based learning with accumulation of uncertain sensory evidence. This model predicted the specific pattern of trial history effects that were found in the data. The results suggest that learning is a critical factor contributing to speed-accuracy tradeoffs in decision-making, and that task history effects are not simply biases but rather the signatures of an optimal learning strategy.
Identifiants
pubmed: 32488065
doi: 10.1038/s41467-020-16196-7
pii: 10.1038/s41467-020-16196-7
pmc: PMC7265464
doi:
Types de publication
Journal Article
Research Support, N.I.H., Extramural
Research Support, Non-U.S. Gov't
Langues
eng
Sous-ensembles de citation
IM
Pagination
2757Subventions
Organisme : NIMH NIH HHS
ID : R01 MH115554
Pays : United States
Références
Roitman, J. D. & Shadlen, M. N. Response of neurons in the lateral intraparietal area during a combined visual discrimination reaction time task. J. Neurosci. 22, 9475–9489 (2002).
pubmed: 12417672
pmcid: 6758024
doi: 10.1523/JNEUROSCI.22-21-09475.2002
Palmer, J., Huk, A. C. & Shadlen, M. N. The effect of stimulus strength on the speed and accuracy of a perceptual decision. J. Vis. 5, 376–404 (2005).
pubmed: 16097871
doi: 10.1167/5.5.1
Chittka, L., Dyer, A. G., Bock, F. & Dornhaus, A. Bees trade off foraging speed for accuracy. Nature 424, 388 (2003).
pubmed: 12879057
doi: 10.1038/424388a
Histed, M. H., Carvalho, L. A. & Maunsell, J. H. R. Psychophysical measurement of contrast sensitivity in the behaving mouse. J. Neurophysiol. 107, 758–765 (2012).
pubmed: 22049334
doi: 10.1152/jn.00609.2011
Bowman, N. E., Kording, K. P. & Gottfried, J. A. Temporal integration of olfactory perceptual evidence in human orbitofrontal cortex. Neuron 75, 916–927 (2012).
pubmed: 22958830
pmcid: 3441053
doi: 10.1016/j.neuron.2012.06.035
Brunton, B. W., Botvinick, M. M. & Brody, C. D. Rats and humans can optimally accumulate evidence for decision-making. Science 340, 95–98 (2013).
pubmed: 23559254
doi: 10.1126/science.1233912
Gold, J. I. & Shadlen, M. N. The neural basis of decision making. Annu. Rev. Neurosci. 30, 535–561 (2007).
pubmed: 17600525
doi: 10.1146/annurev.neuro.29.051605.113038
Ratcliff, R. & McKoon, G. The diffusion decision model: theory and data for two-choice decision tasks. Neural Comput. 20, 873–922 (2008).
pubmed: 18085991
pmcid: 2474742
doi: 10.1162/neco.2008.12-06-420
Beck, J. et al. Probabilistic population codes for Bayesian decision making. Neuron 60, 1142–1152 (2008).
pubmed: 19109917
pmcid: 2742921
doi: 10.1016/j.neuron.2008.09.021
Kiani, R., Hanks, T. D. & Shadlen, M. N. Bounded integration in parietal cortex underlies decisions even when viewing duration is dictated by the environment. J. Neurosci. 28, 3017–3029 (2008).
pubmed: 18354005
pmcid: 6670720
doi: 10.1523/JNEUROSCI.4761-07.2008
Hanks, T. D., Ditterich, J. & Shadlen, M. N. Microstimulation of macaque area LIP affects decision-making in a motion discrimination task. Nat. Neurosci. 9, 682–689 (2006).
pubmed: 16604069
pmcid: 2770004
doi: 10.1038/nn1683
Erlich, J. C., Brunton, B. W., Duan, C. A., Hanks, T. D. & Brody, C. D. Distinct effects of prefrontal and parietal cortex inactivations on an accumulation of evidence task in the rat. eLife 4, e05457 (2015).
Hanks, T. D. et al. Distinct relationships of parietal and prefrontal cortices to evidence accumulation. Nature 520, 220–223 (2015).
pubmed: 25600270
pmcid: 4835184
doi: 10.1038/nature14066
Zariwala, H. A., Kepecs, A., Uchida, N., Hirokawa, J. & Mainen, Z. F. The limits of deliberation in a perceptual decision task. Neuron 78, 339–351 (2013).
pubmed: 23541901
pmcid: 3711252
doi: 10.1016/j.neuron.2013.02.010
Uchida, N. & Mainen, Z. F. Speed and accuracy of olfactory discrimination in the rat. Nat. Neurosci. 6, 1224–1229 (2003).
pubmed: 14566341
doi: 10.1038/nn1142
Rinberg, D., Koulakov, A. & Gelperin, A. Speed-accuracy tradeoff in olfaction. Neuron 51, 351–358 (2006).
pubmed: 16880129
doi: 10.1016/j.neuron.2006.07.013
Abraham, N. M. et al. Maintaining accuracy at the expense of speed: stimulus similarity defines odor discrimination time in mice. Neuron 44, 865–876 (2004).
pubmed: 15572116
Khan, R. M. & Sobel, N. Neural processing at the speed of smell. Neuron 44, 744–747 (2004).
pubmed: 15572105
doi: 10.1016/j.neuron.2004.11.024
Ratcliff, R. A theory of memory retrieval. Psychol. Rev. 85, 59–108 (1978).
doi: 10.1037/0033-295X.85.2.59
Ratcliff, R. & Smith, P. L. A comparison of sequential sampling models for two-choice reaction time. Psychol. Rev. 111, 333–367 (2004).
pubmed: 15065913
pmcid: 1440925
doi: 10.1037/0033-295X.111.2.333
Mulder, M. J., Wagenmakers, E.-J., Ratcliff, R., Boekel, W. & Forstmann, B. U. Bias in the brain: a diffusion model analysis of prior probability and potential payoff. J. Neurosci. 32, 2335–2343 (2012).
pubmed: 22396408
pmcid: 6621823
doi: 10.1523/JNEUROSCI.4156-11.2012
Fründ, I., Wichmann, F. A. & Macke, J. H. Quantifying the effect of intertrial dependence on perceptual decisions. J. Vis. 14, 9 (2014).
pubmed: 24944238
doi: 10.1167/14.7.9
Gold, J. I., Law, C.-T., Connolly, P. & Bennur, S. The relative influences of priors and sensory evidence on an oculomotor decision variable during perceptual learning. J. Neurophysiol. 100, 2653–2668 (2008).
pubmed: 18753326
pmcid: 2585410
doi: 10.1152/jn.90629.2008
Majaj, N. J., Hong, H., Solomon, E. A. & DiCarlo, J. J. Simple learned weighted sums of inferior temporal neuronal firing rates accurately predict human core object recognition performance. J. Neurosci. 35, 13402–13418 (2015).
pubmed: 26424887
pmcid: 4588611
doi: 10.1523/JNEUROSCI.5181-14.2015
Uchida, N., Kepecs, A. & Mainen, Z. F. Seeing at a glance, smelling in a whiff: rapid forms of perceptual decision making. Nat. Rev. Neurosci. 7, 485–491 (2006).
pubmed: 16715056
doi: 10.1038/nrn1933
Rescorla, R. A & Wagner, A R. in Classical Conditioning II Current Research and Theory (eds Black, A. H. & Prokasy, W. F.) Vol. 21, 64–99 (Appleton-Century-Crofts, 1972).
Sutton, R. S. & Barto, A. G. Introduction to Reinforcement Learning (MIT Press, 1998).
Busse, L. et al. The detection of visual contrast in the behaving mouse. J. Neurosci. 31, 11351–11361 (2011).
pubmed: 21813694
pmcid: 6623377
doi: 10.1523/JNEUROSCI.6689-10.2011
Scott, B. B., Constantinople, C. M., Erlich, J. C., Tank, D. W. & Brody, C. D. Sources of noise during accumulation of evidence in unrestrained and voluntarily head-restrained rats. Elife 4, 1–23 (2015).
doi: 10.7554/eLife.11308
Summerfield, C., Behrens, T. E. & Koechlin, E. Perceptual classification in a rapidly changing environment. Neuron 71, 725–736 (2011).
pubmed: 21867887
pmcid: 3975575
doi: 10.1016/j.neuron.2011.06.022
Pouget, A., Drugowitsch, J. & Kepecs, A. Confidence and certainty: distinct probabilistic quantities for different goals. Nat. Neurosci. 19, 366–374 (2016).
pubmed: 26906503
pmcid: 5378479
doi: 10.1038/nn.4240
Drugowitsch, J., Mendonça, A. G., Mainen, Z. F. & Pouget, A. Learning optimal decisions with confidence. Proc. Natl Acad. Sci. USA 116, 24872–24880 (2019).
Beck, J. M., Ma, W. J., Pitkow, X., Latham, P. E. & Pouget, A. Not noisy, just wrong: the role of suboptimal inference in behavioral variability. Neuron 74, 30–39 (2012).
pubmed: 22500627
pmcid: 4486264
doi: 10.1016/j.neuron.2012.03.016
Bogacz, R., Brown, E., Moehlis, J., Holmes, P. & Cohen, J. D. The physics of optimal decision making: a formal analysis of models of performance in two-alternative forced-choice tasks. Psychol. Rev. 113, 700–765 (2006).
pubmed: 17014301
doi: 10.1037/0033-295X.113.4.700
Drugowitsch, J., Moreno-Bote, R., Churchland, A. K., Shadlen, M. N. & Pouget, A. The cost of accumulating evidence in perceptual decision making. J. Neurosci. 32, 3612–3628 (2012).
pubmed: 22423085
pmcid: 3329788
doi: 10.1523/JNEUROSCI.4010-11.2012
Tajima, S., Drugowitsch, J. & Pouget, A. Optimal policy for value-based decision-making. Nat. Commun. 7, 1–12 (2016).
doi: 10.1038/ncomms12400
Stevens, S. S. Psychophysics (Transaction Publishers, 1975).
Wojcik, P. T. & Sirotin, Y. B. Single scale for odor intensity in rat olfaction. Curr. Biol. 24, 568–573 (2015).
doi: 10.1016/j.cub.2014.01.059
Taniguchi, M., Kashiwayanagi, M. & Kurihara, K. Quantitative analysis on odor intensity and quality of optical isomers in turtle olfactory system. Am. J. Physiol. Regul. Integr. Comp. Physiol. 262, R99–R104 (1992).
doi: 10.1152/ajpregu.1992.262.1.R99
Laska, M., Psychologie, M., München, L. & München, D.- Olfactory discrimination ability of human subjects for enantiomers with an isopropenyl group at the chiral center. Chem. Senses 29, 143–152 (2004).
pubmed: 14977811
doi: 10.1093/chemse/bjh019
Pierce, J. D., Zeng, X.-N., Aronov, E. V., Preti, G. & Wysocki, C. J. Cross-adaptation of sweaty-smelling 3-methyl-2- hexenoic acid by a structurally-similar, pleasant-smelling odorant. Chem. Senses 20, 401–411 (1995).
pubmed: 8590025
doi: 10.1093/chemse/20.4.401
Churchland, A. K. et al. Variance as a signature of neural computations during decision making. Neuron 69, 818–831 (2011).
pubmed: 21338889
pmcid: 3066020
doi: 10.1016/j.neuron.2010.12.037
Herrnstein, R. J. Formal properties of the matching law. J. Exp. Anal. Behav. 21, 159–164 (1974).
pubmed: 16811728
pmcid: 1333179
doi: 10.1901/jeab.1974.21-159
Baum, W. M. Matching, undermatching, and overmatching in studies of choice. J. Exp. Anal. Behav. 32, 269–281 (1979).
pubmed: 501274
pmcid: 1332902
doi: 10.1901/jeab.1979.32-269
Sugrue, L. P., Corrado, G. S. & Newsome, W. T. Matching behavior and the representation of value in the parietal cortex. Science 304, 1782–1787 (2004).
pubmed: 15205529
doi: 10.1126/science.1094765
Lauwereyns, J., Watanabe, K., Coe, B. & Hikosaka, O. A neural correlate of response bias in monkey caudate nucleus. Nature 418, 413–417 (2002).
pubmed: 12140557
pmcid: 12140557
doi: 10.1038/nature00892
Roesch, M. R. & Olson, C. R. Neuronal activity related to reward value and motivation in primate frontal cortex. Science 304, 307–310 (2004).
pubmed: 15073380
doi: 10.1126/science.1093223
Kiani, R. & Shadlen, M. N. Representation of confidence associated with a decision by neurons in the parietal cortex. Science 324, 759–764 (2009).
pubmed: 19423820
pmcid: 2738936
doi: 10.1126/science.1169405
Usher, M. & McClelland, J. L. The time course of perceptual choice: the leaky, competing accumulator model. Psychol. Rev. 108, 550–592 (2001).
pubmed: 11488378
doi: 10.1037/0033-295X.108.3.550
Cockburn, J., Collins, A. G. E. & Frank, M. J. A reinforcement learning mechanism responsible for the valuation of free choice. Neuron 83, 551–557 (2014).
pubmed: 25066083
pmcid: 4126879
doi: 10.1016/j.neuron.2014.06.035
Frank, M. J. et al. FMRI and EEG predictors of dynamic decision parameters during human reinforcement learning. J. Neurosci. 35, 485–494 (2015).
pubmed: 25589744
pmcid: 4293405
doi: 10.1523/JNEUROSCI.2036-14.2015
Collins, A. G. E., Albrecht, M. A., Waltz, J. A., Gold, J. M. & Frank, M. J. Interactions among working memory, reinforcement learning, and effort in value-based choice: a new paradigm and selective deficits in schizophrenia. Biol. Psychiatry 82, 431–439 (2017).
pubmed: 28651789
pmcid: 5573149
doi: 10.1016/j.biopsych.2017.05.017
Drugowitsch, J. Fast and accurate Monte Carlo sampling of first-passage times from Wiener diffusion models. Sci. Rep. 6, 1–13 (2016).
doi: 10.1038/srep20490
Simpson, D. P., Turner, I. W. & Pettitt, A. N. Sampling from Gaussian Markov random fields conditioned on linear constraints. ANZIAM J. 48, 1041 (2008).
doi: 10.21914/anziamj.v48i0.131
Wichmann, F. & Hill, N. J. The psychometric function: I. Fitting, sampling, and goodness of fit. Percept. Psychophys. 63, 1293–1313 (2001).
pubmed: 11800458
doi: 10.3758/BF03194544
Schwarz, G. Estimating the dimension of a model. Ann. Stat. 6, 461–464 (1978).
doi: 10.1214/aos/1176344136
Wit, E., Heuvel, Evanden & Romeijn, J.-W. ‘All models are wrong.’: an introduction to model uncertainty. Stat. Neerl. 66, 217–236 (2012).
doi: 10.1111/j.1467-9574.2012.00530.x