Distinct basal ganglia contributions to learning from implicit and explicit value signals in perceptual decision-making.
Journal
Nature communications
ISSN: 2041-1723
Titre abrégé: Nat Commun
Pays: England
ID NLM: 101528555
Informations de publication
Date de publication:
22 Jun 2024
22 Jun 2024
Historique:
received:
07
09
2023
accepted:
07
06
2024
medline:
23
6
2024
pubmed:
23
6
2024
entrez:
22
6
2024
Statut:
epublish
Résumé
Metacognitive evaluations of confidence provide an estimate of decision accuracy that could guide learning in the absence of explicit feedback. We examine how humans might learn from this implicit feedback in direct comparison with that of explicit feedback, using simultaneous EEG-fMRI. Participants performed a motion direction discrimination task where stimulus difficulty was increased to maintain performance, with intermixed explicit- and no-feedback trials. We isolate single-trial estimates of post-decision confidence using EEG decoding, and find these neural signatures re-emerge at the time of feedback together with separable signatures of explicit feedback. We identified these signatures of implicit versus explicit feedback along a dorsal-ventral gradient in the striatum, a finding uniquely enabled by an EEG-fMRI fusion. These two signals appear to integrate into an aggregate representation in the external globus pallidus, which could broadcast updates to improve cortical decision processing via the thalamus and insular cortex, irrespective of the source of feedback.
Identifiants
pubmed: 38909014
doi: 10.1038/s41467-024-49538-w
pii: 10.1038/s41467-024-49538-w
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
5317Subventions
Organisme : RCUK | Economic and Social Research Council (ESRC)
ID : ES/L012995/1
Organisme : EC | EU Framework Programme for Research and Innovation H2020 | H2020 Priority Excellent Science | H2020 European Research Council (H2020 Excellent Science - European Research Council)
ID : 865003
Informations de copyright
© 2024. The Author(s).
Références
Watanabe, T. & Sasaki, Y. Perceptual learning: toward a comprehensive theory. Annu. Rev. Psychol. 66, 197–221 (2015).
pubmed: 25251494
doi: 10.1146/annurev-psych-010814-015214
Dosher, B. A. & Lu, Z.-L. Perceptual learning reflects external noise filtering and internal noise reduction through channel reweighting. Proc. Natl Acad. Sci. USA 95, 13988–13993 (1998).
pubmed: 9811913
pmcid: 25004
doi: 10.1073/pnas.95.23.13988
Law, C.-T. & Gold, J. I. Reinforcement learning can account for associative and perceptual learning on a visual-decision task. Nat. Neurosci. 12, 655–663 (2009).
pubmed: 19377473
pmcid: 2674144
doi: 10.1038/nn.2304
Herzog, M. H. & Fahle, M. The role of feedback in learning a vernier discrimination task. Vis. Res. 37, 2133–2141 (1997).
pubmed: 9327060
doi: 10.1016/S0042-6989(97)00043-6
Karni, A. & Sagi, D. Where practice makes perfect in texture discrimination: evidence for primary visual cortex plasticity. Proc. Natl Acad. Sci. USA 88, 4966–4970 (1991).
pubmed: 2052578
pmcid: 51788
doi: 10.1073/pnas.88.11.4966
Watanabe, T., Nanez, J. E. & Sasaki, Y. Perceptual learning without perception. Nature 413, 844–848 (2001).
pubmed: 11677607
doi: 10.1038/35101601
Pouget, A., Drugowitsch, J. & Kepecs, A. Confidence and certainty: distinct probabilistic quantities for different goals. Nat. Neurosci. 19, 366–374 (2016).
pubmed: 26906503
pmcid: 5378479
doi: 10.1038/nn.4240
Yeung, N. & Summerfield, C. Metacognition in human decision-making: confidence and error monitoring. Philos. Trans. R. Soc. B Biol. Sci. 367, 1310–1321 (2012).
doi: 10.1098/rstb.2011.0416
Sutton, R. S. & Barto, A. G. Introduction to Reinforcement Learning. Vol. 135 (MIT Press, 1998).
Diaz, J. A., Queirazza, F. & Philiastides, M. G. Perceptual learning alters post-sensory processing in human decision-making. Nat. Hum. Behav. 1, 0035 (2017).
doi: 10.1038/s41562-016-0035
Daniel, R. & Pollmann, S. Striatal activations signal prediction errors on confidence in the absence of external feedback. Neuroimage 59, 3457–3467 (2012).
pubmed: 22146752
doi: 10.1016/j.neuroimage.2011.11.058
Guggenmos, M., Wilbertz, G., Hebart, M. N. & Sterzer, P. Mesolimbic confidence signals guide perceptual learning in the absence of external feedback. eLife 5, e13388 (2016).
pubmed: 27021283
pmcid: 4821804
doi: 10.7554/eLife.13388
Drugowitsch, J., Mendonça, A. G., Mainen, Z. F. & Pouget, A. Learning optimal decisions with confidence. Proc. Natl Acad. Sci. USA 116, 24872–24880 (2019).
pubmed: 31732671
pmcid: 6900530
doi: 10.1073/pnas.1906787116
Hogarth, R. M. in Information Sampling and Adaptive Cognition Vol. 1 (eds. Fiedler, K. & Juslin, P.) Ch. 19 (Cambridge Univ. Press, 2006).
Bang, J. W., Shekhar, M. & Rahnev, D. Sensory noise increases metacognitive efficiency. J. Exp. Psychol. Gen. 148, 437 (2019).
pubmed: 30382720
doi: 10.1037/xge0000511
Baranski, J. V. & Petrusic, W. M. The calibration and resolution of confidence in perceptual judgments. Percept. Psychophys. 55, 412–428 (1994).
pubmed: 8036121
doi: 10.3758/BF03205299
Shekhar, M. & Rahnev, D. Sources of metacognitive inefficiency. Trends Cogn. Sci. 25, 12–23 (2021).
pubmed: 33214066
doi: 10.1016/j.tics.2020.10.007
Zylberberg, A., Roelfsema, P. R. & Sigman, M. Variance misperception explains illusions of confidence in simple perceptual decisions. Conscious. Cogn. 27, 246–253 (2014).
pubmed: 24951943
doi: 10.1016/j.concog.2014.05.012
Zhang, H. & Maloney, L. T. Ubiquitous log odds: a common representation of probability and frequency distortion in perception, action, and cognition. Front. Neurosci. 6, 1 (2012).
pubmed: 22294978
pmcid: 3261445
doi: 10.3389/fnins.2012.00001
Geurts, L. S., Cooke, J. R. H., van Bergen, R. S. & Jehee, J. F. M. Subjective confidence reflects representation of Bayesian probability in cortex. Nat. Hum. Behav. 6, 294–305 (2022).
pubmed: 35058641
pmcid: 7612428
doi: 10.1038/s41562-021-01247-w
Frömer, R. et al. Response-based outcome predictions and confidence regulate feedback processing and learning. eLife 10, e62825 (2021).
pubmed: 33929323
pmcid: 8121545
doi: 10.7554/eLife.62825
Joshua, M., Adler, A. & Bergman, H. The dynamics of dopamine in control of motor behavior. Curr. Opin. Neurobiol. 19, 615–620 (2009).
pubmed: 19896833
doi: 10.1016/j.conb.2009.10.001
Schultz, W. Updating dopamine reward signals. Curr. Opin. Neurobiol. 23, 229–238 (2013).
pubmed: 23267662
pmcid: 3866681
doi: 10.1016/j.conb.2012.11.012
Abler, B., Walter, H., Erk, S., Kammerer, H. & Spitzer, M. Prediction error as a linear function of reward probability is coded in human nucleus accumbens. Neuroimage 31, 790–795 (2006).
pubmed: 16487726
doi: 10.1016/j.neuroimage.2006.01.001
Samejima, K., Ueda, Y., Doya, K. & Kimura, M. Representation of action-specific reward values in the striatum. Science 310, 1337–1340 (2005).
pubmed: 16311337
doi: 10.1126/science.1115270
Ito, M. & Doya, K. Parallel representation of value-based and finite state-based strategies in the ventral and dorsal striatum. PLoS Comput. Biol. 11, e1004540 (2015).
pubmed: 26529522
pmcid: 4631489
doi: 10.1371/journal.pcbi.1004540
Schultz, W. Dopamine reward prediction error coding. Dialogues Clin. Neurosci. 18, 23–32 (2016).
pubmed: 27069377
pmcid: 4826767
doi: 10.31887/DCNS.2016.18.1/wschultz
Schultz, W., Dayan, P. & Montague, P. R. A neural substrate of prediction and reward. Science 275, 1593–1599 (1997).
pubmed: 9054347
doi: 10.1126/science.275.5306.1593
O’Doherty, J. P., Buchanan, T. W., Seymour, B. & Dolan, R. J. Predictive neural coding of reward preference involves dissociable responses in human ventral midbrain and ventral striatum. Neuron 49, 157–166 (2006).
pubmed: 16387647
doi: 10.1016/j.neuron.2005.11.014
Hebart, M. N., Schriever, Y., Donner, T. H. & Haynes, J.-D. The relationship between perceptual decision variables and confidence in the human brain. Cereb. Cortex 26, 118–130 (2016).
pubmed: 25112281
doi: 10.1093/cercor/bhu181
Ding, L. & Gold, J. I. Caudate encodes multiple computations for perceptual decisions. J. Neurosci. 30, 15747–15759 (2010).
pubmed: 21106814
pmcid: 3005761
doi: 10.1523/JNEUROSCI.2894-10.2010
Costa, R. M. Plastic corticostriatal circuits for action learning: what’s dopamine got to do with it? Ann. N. Y. Acad. Sci. 1104, 172–191 (2007).
pubmed: 17435119
doi: 10.1196/annals.1390.015
Horga, G. et al. Changes in corticostriatal connectivity during reinforcement learning in humans. Hum. Brain Mapp. 36, 793–803 (2015).
pubmed: 25393839
doi: 10.1002/hbm.22665
Maia, T. V. Reinforcement learning, conditioning, and the brain: successes and challenges. Cogn. Affect. Behav. Neurosci. 9, 343–364 (2009).
pubmed: 19897789
doi: 10.3758/CABN.9.4.343
Liljeholm, M. & O’Doherty, J. P. Contributions of the striatum to learning, motivation, and performance: an associative account. Trends Cogn. Sci. 16, 467–475 (2012).
pubmed: 22890090
pmcid: 3449003
doi: 10.1016/j.tics.2012.07.007
Cox, J. & Witten, I. B. Striatal circuits for reward learning and decision-making. Nat. Rev. Neurosci. 20, 482–494 (2019).
pubmed: 31171839
pmcid: 7231228
doi: 10.1038/s41583-019-0189-2
Julesz, B. Foundations of Cyclopean Perception (1971).
Green, D. M. & Swets, J. A. Signal Detection Theory and Psychophysics. Vol. 1 (Wiley New York, 1966).
Rosa, M. J., Daunizeau, J. & Friston, K. J. EEG-fMRI integration: a critical review of biophysical modeling and data analysis approaches. J. Integr. Neurosci. 09, 453–476 (2010).
doi: 10.1142/S0219635210002512
Philiastides, M. G., Tu, T. & Sajda, P. Inferring macroscale brain dynamics via fusion of simultaneous EEG-fMRI. Annu. Rev. Neurosci. 44, 315–334 (2021).
pubmed: 33761268
doi: 10.1146/annurev-neuro-100220-093239
Gherman, S. & Philiastides, M. G. Human VMPFC encodes early signatures of confidence in perceptual decisions. eLife 7, e38293 (2018).
pubmed: 30247123
pmcid: 6199131
doi: 10.7554/eLife.38293
Gherman, S. & Philiastides, M. G. Neural representations of confidence emerge from the process of decision formation during perceptual choices. Neuroimage 106, 134–143 (2015).
pubmed: 25463461
doi: 10.1016/j.neuroimage.2014.11.036
Pereira, M. et al. Disentangling the origins of confidence in speeded perceptual judgments through multimodal imaging. Proc. Natl Acad. Sci. USA 117, 8382–8390 (2020).
pubmed: 32238562
pmcid: 7165419
doi: 10.1073/pnas.1918335117
Murphy, P. R., Robertson, I. H., Harty, S. & O’Connell, R. G. Neural evidence accumulation persists after choice to inform metacognitive judgments. eLife 4, e11946 (2015).
pubmed: 26687008
pmcid: 4749550
doi: 10.7554/eLife.11946
Vaccaro, A. G. & Fleming, S. M. Thinking about thinking: a coordinate-based meta-analysis of neuroimaging studies of metacognitive judgements. Brain Neurosci. Adv. 2, 239821281881059 (2018).
doi: 10.1177/2398212818810591
Rouault, M., Lebreton, M. & Pessiglione, M. A shared brain system forming confidence judgment across cognitive domains. Cereb. Cortex 33, 1426–1439 (2023).
pubmed: 35552662
doi: 10.1093/cercor/bhac146
Liu, T. & Pleskac, T. J. Neural correlates of evidence accumulation in a perceptual decision task. J. Neurophysiol. 106, 2383–2398 (2011).
pubmed: 21849612
doi: 10.1152/jn.00413.2011
Clithero, J. A. & Rangel, A. Informatic parcellation of the network involved in the computation of subjective value. Soc. Cogn. Affect. Neurosci. 9, 1289–1302 (2014).
pubmed: 23887811
doi: 10.1093/scan/nst106
Rangel, A. & Hare, T. Neural computations associated with goal-directed choice. Curr. Opin. Neurobiol. 20, 262–270 (2010).
pubmed: 20338744
doi: 10.1016/j.conb.2010.03.001
Philiastides, M. G., Biele, G. & Heekeren, H. R. A mechanistic account of value computation in the human brain. Proc. Natl Acad. Sci. USA 107, 9430–9435 (2010).
pubmed: 20439711
pmcid: 2889112
doi: 10.1073/pnas.1001732107
Chiu, Y.-C., Jiang, J. & Egner, T. The caudate nucleus mediates learning of stimulus–control state associations. J. Neurosci. 37, 1028–1038 (2017).
pubmed: 28123033
doi: 10.1523/JNEUROSCI.0778-16.2016
Doi, T., Fan, Y., Gold, J. I. & Ding, L. The caudate nucleus contributes causally to decisions that balance reward and uncertain visual information. Elife 9, e56694 (2020).
pubmed: 32568068
pmcid: 7308093
doi: 10.7554/eLife.56694
Smith, Y., Bevan, M. D., Shink, E. & Bolam, J. P. Microcircuitry of the direct and indirect pathways of the basal ganglia. Neuroscience 86, 353–387 (1998).
pubmed: 9881853
Dong, J., Hawes, S., Wu, J., Le, W. & Cai, H. Connectivity and functionality of the globus pallidus externa under normal conditions and Parkinson’s disease. Front. Neural Circuits 15, 645287 (2021).
pubmed: 33737869
pmcid: 7960779
doi: 10.3389/fncir.2021.645287
Fiore, V. G. et al. Value encoding in the globus pallidus: fMRI reveals an interaction effect between reward and dopamine drive. Neuroimage 173, 249–257 (2018).
pubmed: 29481966
doi: 10.1016/j.neuroimage.2018.02.048
Verdonck, S., Loossens, T. & Philiastides, M. G. The Leaky Integrating Threshold and its impact on evidence accumulation models of choice response time (RT). Psychol. Rev. 128, 203–221 (2021).
pubmed: 32915011
doi: 10.1037/rev0000258
Balsdon, T., Verdonck, S., Loossens, T. & Philiastides, M. G. Secondary motor integration as a final arbiter in sensorimotor decision-making. PLOS Biol. 21, e3002200 (2023).
pubmed: 37459392
pmcid: 10393169
doi: 10.1371/journal.pbio.3002200
Yin, H. H., Knowlton, B. J. & Balleine, B. W. Lesions of dorsolateral striatum preserve outcome expectancy but disrupt habit formation in instrumental learning. Eur. J. Neurosci. 19, 181–189 (2004).
pubmed: 14750976
doi: 10.1111/j.1460-9568.2004.03095.x
Sala-Bayo, J. et al. Dorsal and ventral striatal dopamine D1 and D2 receptors differentially modulate distinct phases of serial visual reversal learning. Neuropsychopharmacology 45, 736–744 (2020).
pubmed: 31940660
pmcid: 7075980
doi: 10.1038/s41386-020-0612-4
Cataldi, S., Stanley, A. T., Miniaci, M. C. & Sulzer, D. Interpreting the role of the striatum during multiple phases of motor learning. FEBS J. 289, 2263–2281 (2022).
pubmed: 33977645
doi: 10.1111/febs.15908
Gordon, E. M. et al. Individualized functional subnetworks connect human striatum and frontal cortex. Cereb. Cortex 32, 2868–2884 (2022).
pubmed: 34718460
doi: 10.1093/cercor/bhab387
Averbeck, B. B. & Murray, E. A. Hypothalamic interactions with large-scale neural circuits underlying reinforcement learning and motivated behavior. Trends Neurosci. 43, 681–694 (2020).
pubmed: 32762959
pmcid: 7483858
doi: 10.1016/j.tins.2020.06.006
De Martino, B., Fleming, S. M., Garrett, N. & Dolan, R. J. Confidence in value-based choice. Nat. Neurosci. 16, 105–110 (2013).
pubmed: 23222911
doi: 10.1038/nn.3279
Lebreton, M., Abitbol, R., Daunizeau, J. & Pessiglione, M. Automatic integration of confidence in the brain valuation signal. Nat. Neurosci. 18, 1159–1167 (2015).
pubmed: 26192748
doi: 10.1038/nn.4064
Alexander, G. E. & Crutcher, M. D. Functional architecture of basal ganglia circuits: neural substrates of parallel processing. Trends Neurosci. 13, 266–271 (1990).
pubmed: 1695401
doi: 10.1016/0166-2236(90)90107-L
Bolam, J. P., Hanley, J. J., Booth, P. A. C. & Bevan, M. D. Synaptic organisation of the basal ganglia. J. Anat. 196, 527–542 (2000).
pubmed: 10923985
pmcid: 1468095
doi: 10.1046/j.1469-7580.2000.19640527.x
Gittis, A. H. et al. New roles for the external globus pallidus in basal ganglia circuits and behavior. J. Neurosci. 34, 15178–15183 (2014).
pubmed: 25392486
pmcid: 4228126
doi: 10.1523/JNEUROSCI.3252-14.2014
Forstmann, B. U. et al. Cortico-striatal connections predict control over speed and accuracy in perceptual decision making. Proc. Natl Acad. Sci. USA 107, 15916–15920 (2010).
pubmed: 20733082
pmcid: 2936628
doi: 10.1073/pnas.1004932107
Bogacz, R., Martin Moraud, E., Abdi, A., Magill, P. J. & Baufreton, J. Properties of neurons in external globus pallidus can support optimal action selection. PLoS Comput. Biol. 12, e1005004 (2016).
pubmed: 27389780
pmcid: 4936724
doi: 10.1371/journal.pcbi.1005004
Lilascharoen, V. et al. Divergent pallidal pathways underlying distinct Parkinsonian behavioral deficits. Nat. Neurosci. 24, 504–515 (2021).
pubmed: 33723433
pmcid: 8907079
doi: 10.1038/s41593-021-00810-y
Saga, Y., Hoshi, E. & Tremblay, L. Roles of multiple globus pallidus territories of monkeys and humans in motivation, cognition and action: an anatomical, physiological and pathophysiological review. Front. Neuroanat. 11, 30 (2017).
pubmed: 28442999
pmcid: 5385466
doi: 10.3389/fnana.2017.00030
Delorme, A. & Makeig, S. EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. J. Neurosci. Methods 134, 9–21 (2004).
pubmed: 15102499
doi: 10.1016/j.jneumeth.2003.10.009
Fouragnan, E., Retzler, C., Mullinger, K. & Philiastides, M. G. Two spatiotemporally distinct value systems shape reward-based learning in the human brain. Nat. Commun. 6, 8107 (2015).
pubmed: 26348160
doi: 10.1038/ncomms9107
Fouragnan, E., Queirazza, F., Retzler, C., Mullinger, K. J. & Philiastides, M. G. Spatiotemporal neural characterization of prediction error valence and surprise during reward learning in humans. Sci. Rep. 7, 1–18 (2017).
doi: 10.1038/s41598-017-04507-w
Arabadzhiyska, D. H. et al. A common neural account for social and nonsocial decisions. J. Neurosci. 42, 9030–9044 (2022).
pubmed: 36280264
pmcid: 9732824
doi: 10.1523/JNEUROSCI.0375-22.2022
Smith, S. M. et al. Advances in functional and structural MR image analysis and implementation as FSL. Neuroimage 23, S208–S219 (2004).
pubmed: 15501092
doi: 10.1016/j.neuroimage.2004.07.051
Smith, S. M. Fast robust automated brain extraction. Hum. Brain Mapp. 17, 143–155 (2002).
pubmed: 12391568
pmcid: 6871816
doi: 10.1002/hbm.10062
Woolrich, M. W., Ripley, B. D., Brady, M. & Smith, S. M. Temporal autocorrelation in univariate linear modeling of FMRI data. Neuroimage 14, 1370–1386 (2001).
pubmed: 11707093
doi: 10.1006/nimg.2001.0931
Woolrich, M. W., Behrens, T. E., Beckmann, C. F., Jenkinson, M. & Smith, S. M. Multilevel linear modelling for FMRI group analysis using Bayesian inference. Neuroimage 21, 1732–1747 (2004).
pubmed: 15050594
doi: 10.1016/j.neuroimage.2003.12.023
Jenkinson, M., Bannister, P., Brady, M. & Smith, S. Improved optimization for the robust and accurate linear registration and motion correction of brain images. Neuroimage 17, 825–841 (2002).
pubmed: 12377157
doi: 10.1006/nimg.2002.1132
Rescorla, R. A. A theory of Pavlovian conditioning: variations in the effectiveness of reinforcement and non-reinforcement. Class. Cond. Curr. Res. Theory 2, 64–69 (1972).
Mamassian, P. Visual confidence. Annu. Rev. Vis. Sci. 2, 459–481 (2016).
pubmed: 28532359
doi: 10.1146/annurev-vision-111815-114630
Parra, L. C., Spence, C. D., Gerson, A. D. & Sajda, P. Recipes for the linear analysis of EEG. Neuroimage 28, 326–341 (2005).
pubmed: 16084117
doi: 10.1016/j.neuroimage.2005.05.032
Jordan, M. I. & Jacobs, R. A. Hierarchical mixtures of experts and the EM algorithm. Neural Comput. 6, 181–214 (1994).
doi: 10.1162/neco.1994.6.2.181
Sajda, P., Gerson, A. D., Philiastides, M. G. & Parra, L. C. Single-trial analysis of EEG during rapid visual discrimination: enabling cortically-coupled computer vision. Brain-Comput. Interfacing 423, 44 (2007).
Maris, E. & Oostenveld, R. Nonparametric statistical testing of EEG- and MEG-data. J. Neurosci. Methods 164, 177–190 (2007).
pubmed: 17517438
doi: 10.1016/j.jneumeth.2007.03.024
O’Reilly, J. X., Woolrich, M. W., Behrens, T. E., Smith, S. M. & Johansen-Berg, H. Tools of the trade: psychophysiological interactions and functional connectivity. Soc. Cogn. Affect. Neurosci. 7, 604–609 (2012).
pubmed: 22569188
pmcid: 3375893
doi: 10.1093/scan/nss055