The neural correlates of continuous feedback processing.
EEG
ERP
dopamine
feedback
reward
stimulus-preceding negativity (SPN)
Journal
Psychophysiology
ISSN: 1540-5958
Titre abrégé: Psychophysiology
Pays: United States
ID NLM: 0142657
Informations de publication
Date de publication:
12 2023
12 2023
Historique:
revised:
12
07
2023
received:
15
11
2022
accepted:
12
07
2023
medline:
8
11
2023
pubmed:
24
7
2023
entrez:
24
7
2023
Statut:
ppublish
Résumé
Feedback processing is commonly studied by analyzing the brain's response to discrete rather than continuous events. Such studies have led to the hypothesis that rapid phasic midbrain dopaminergic activity tracks reward prediction errors (RPEs), the effects of which are measurable at the scalp via electroencephalography (EEG). Although studies using continuous feedback are sparse, recent animal work suggests that moment-to-moment changes in reward are tracked by slowly ramping midbrain dopaminergic activity. Some have argued that these ramping signals index state values rather than RPEs. Our goal here was to develop an EEG measure of continuous feedback processing in humans, then test whether its behavior could be accounted for by the RPE hypothesis. Participants completed a stimulus-response learning task in which a continuous reward cue gradually increased or decreased over time. A regression-based unmixing approach revealed EEG activity with a topography and time course consistent with the stimulus-preceding negativity (SPN), a scalp potential previously linked to reward anticipation and tonic dopamine release. Importantly, this reward-related activity depended on outcome expectancy: as predicted by the RPE hypothesis, activity for expected reward cues was reduced compared to unexpected reward cues. These results demonstrate the possibility of using human scalp-recorded potentials to track continuous feedback processing, and test candidate hypotheses of this activity.
Types de publication
Journal Article
Research Support, Non-U.S. Gov't
Langues
eng
Sous-ensembles de citation
IM
Pagination
e14399Subventions
Organisme : Wellcome Trust
ID : 203139/Z/16/Z
Pays : United Kingdom
Organisme : Department of Health
Pays : United Kingdom
Informations de copyright
© 2023 The Authors. Psychophysiology published by Wiley Periodicals LLC on behalf of Society for Psychophysiological Research.
Références
Brainard, D. H. (1997). The psychophysics toolbox. Spatial Vision, 10, 433-436.
Brunia, C. H. M., van Boxtel, G. J. M., & Böcker, K. B. E. (2011). Negative slow waves as indices of anticipation: The Bereitschaftspotential, the contingent negative variation, and the stimulus-preceding negativity. In E. S. Kappenman & S. J. Luck (Eds.), The Oxford handbook of event-related potential components. Oxford University Press. https://doi.org/10.1093/oxfordhb/9780195374148.013.0108
Burns, M. D., Bigdely-Shamlo, N., Smith, N. J., Kreutz-Delgado, K., & Makeig, S. (2013). Comparison of averaging and regression techniques for estimating event related potentials. 2013 35th annual international conference of the IEEE engineering in medicine and biology society (EMBC), 1680-1683 https://doi.org/10.1109/EMBC.2013.6609841
Catena, A., Perales, J. C., Megías, A., Cándido, A., Jara, E., & Maldonado, A. (2012). The brain network of expectancy and uncertainty processing. PLoS One, 7(7), e40252. https://doi.org/10.1371/journal.pone.0040252
Crosse, M. J., Di Liberto, G. M., Bednar, A., & Lalor, E. C. (2016). The multivariate temporal response function (mTRF) toolbox: A MATLAB toolbox for relating neural signals to continuous stimuli. Frontiers in Human Neuroscience, 10, 1-14. https://doi.org/10.3389/fnhum.2016.00604
Cumming, G. (2014). The new statistics: Why and how. Psychological Science, 25(1), 7-29. https://doi.org/10.1177/0956797613504966
Delorme, A., & Makeig, S. (2004). EEGLAB: An open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. Journal of Neuroscience Methods, 134(1), 9-21. https://doi.org/10.1016/j.jneumeth.2003.10.009
Ehinger, B. V., & Dimigen, O. (2019). Unfold: An integrated toolbox for overlap correction, non-linear modeling, and regression-based EEG analysis. PeerJ, 7, e7838. https://doi.org/10.7717/peerj.7838
Fiorillo, C. D., Tobler, P. N., & Schultz, W. (2003). Discrete coding of reward probability and uncertainty by dopamine neurons. Science, 299(5614), 1898-1902. https://doi.org/10.1126/science.1077349
Foti, D., & Hajcak, G. (2012). Genetic variation in dopamine moderates neural response during reward anticipation and delivery: Evidence from event-related potentials. Psychophysiology, 49(5), 617-626. https://doi.org/10.1111/j.1469-8986.2011.01343.x
Fuentemilla, L., Cucurell, D., Marco-Pallarés, J., Guitart-Masip, M., Morís, J., & Rodríguez-Fornells, A. (2013). Electrophysiological correlates of anticipating improbable but desired events. NeuroImage, 78, 135-144. https://doi.org/10.1016/j.neuroimage.2013.03.062
Glazer, J. E., Kelley, N. J., Pornpattananangkul, N., Mittal, V. A., & Nusslock, R. (2018). Beyond the FRN: Broadening the time-course of EEG and ERP components implicated in reward processing. International Journal of Psychophysiology, 132, 184-202. https://doi.org/10.1016/j.ijpsycho.2018.02.002
Hamid, A. A., Pettibone, J. R., Mabrouk, O. S., Hetrick, V. L., Schmidt, R., Vander Weele, C. M., Kennedy, R. T., Aragona, B. J., & Berke, J. D. (2016). Mesolimbic dopamine signals the value of work. Nature Neuroscience, 19(1), 117-126. Article 1. https://doi.org/10.1038/nn.4173
Hassall, C. D., Harley, J., Kolling, N., & Hunt, L. T. (2022). Temporal scaling of human scalp-recorded potentials (p. 2020.12.11.421180). bioRxiv https://doi.org/10.1101/2020.12.11.421180
Holroyd, C. B., & Coles, M. G. (2002). The neural basis of human error processing: Reinforcement learning, dopamine, and the error-related negativity. Psychological Review, 109(4), 679-709. https://doi.org/10.1037/0033-295X.109.4.679
Holroyd, C. B., & Krigolson, O. E. (2007). Reward prediction error signals associated with a modified time estimation task. Psychophysiology, 44(6), 913-917. https://doi.org/10.1111/j.1469-8986.2007.00561.x
Jonas, D. (2008). Violin Plots for plotting multiple distributions (distributionPlot.m). https://uk.mathworks.com/matlabcentral/fileexchange/23661-violin-plots-for-plotting-multiple-distributions-distributionplot-m
Kim, H. R., Malik, A. N., Mikhael, J. G., Bech, P., Tsutsui-Kimura, I., Sun, F., Zhang, Y., Li, Y., Watabe-Uchida, M., Gershman, S. J., & Uchida, N. (2020). A unified framework for dopamine signals across timescales. Cell, 183(6), 1600-1616.e25. https://doi.org/10.1016/j.cell.2020.11.013
Kotani, Y., Kishida, S., Hiraku, S., Suda, K., Ishii, M., & Aihara, Y. (2003). Effects of information and reward on stimulus-preceding negativity prior to feedback stimuli. Psychophysiology, 40(5), 818-826. https://doi.org/10.1111/1469-8986.00082
Krigolson, O. E. (2017). Event-related brain potentials and the study of reward processing: Methodological considerations. International Journal of Psychophysiology, 132, 175-183. https://doi.org/10.1016/j.ijpsycho.2017.11.007
Kristensen, E., Guerin-Dugué, A., & Rivet, B. (2017). Regularization and a general linear model for event-related potential estimation. Behavior Research Methods, 49(6), 2255-2274. https://doi.org/10.3758/s13428-017-0856-z
Lopez-Calderon, J., & Luck, S. J. (2014). ERPLAB: An open-source toolbox for the analysis of event-related potentials. Frontiers in Human Neuroscience, 8, 1-14. https://doi.org/10.3389/fnhum.2014.00213
Luck, S. J. (2014). A closer look at ERPs and ERP components. In An introduction to the event-related potential technique (2nd ed., pp. 35-70). The MIT Press.
Maris, E., & Oostenveld, R. (2007). Nonparametric statistical testing of EEG- and MEG-data. Journal of Neuroscience Methods, 164(1), 177-190. https://doi.org/10.1016/j.jneumeth.2007.03.024
Mattox, S. T., Valle-Inclán, F., & Hackley, S. A. (2006). Psychophysiological evidence for impaired reward anticipation in Parkinson's disease. Clinical Neurophysiology, 117(10), 2144-2153. https://doi.org/10.1016/j.clinph.2006.05.026
Mikhael, J. G., Kim, H. R., Uchida, N., & Gershman, S. J. (2022). The role of state uncertainty in the dynamics of dopamine. Current Biology, 32(5), 1077-1087.e9. https://doi.org/10.1016/j.cub.2022.01.025
Mohebi, A., Pettibone, J. R., Hamid, A. A., Wong, J.-M. T., Vinson, L. T., Patriarchi, T., Tian, L., Kennedy, R. T., & Berke, J. D. (2019). Dissociable dopamine dynamics for learning and motivation. Nature, 570(7759), 65-70. Article 7759. https://doi.org/10.1038/s41586-019-1235-y
Morís, J., Luque, D., & Rodríguez-Fornells, A. (2013). Learning-induced modulations of the stimulus-preceding negativity. Psychophysiology, 50(9), 931-939. https://doi.org/10.1111/psyp.12073
Oostenveld, R., Fries, P., Maris, E., & Schoffelen, J.-M. (2010). FieldTrip: Open source software for advanced analysis of MEG, EEG, and invasive electrophysiological data. Computational Intelligence and Neuroscience, 2011, 1-9. https://doi.org/10.1155/2011/156869
Pelli, D. G. (1997). The VideoToolbox software for visual psychophysics: Transforming numbers into movies. Spatial Vision, 10(4), 437-442. https://doi.org/10.1163/156856897X00366
Pion-Tonachini, L., Kreutz-Delgado, K., & Makeig, S. (2019). ICLabel: An automated electroencephalographic independent component classifier, dataset, and website. NeuroImage, 198, 181-197. https://doi.org/10.1016/j.neuroimage.2019.05.026
Proudfit, G. H. (2015). The reward positivity: From basic research on reward to a biomarker for depression. Psychophysiology, 52(4), 449-459. https://doi.org/10.1111/psyp.12370
Reichel, L., & Ye, Q. (2008). Simple square smoothing regularization operators. Electronic Transactions on Numerical Analysis, 33, 63-83.
Ruesseler, M., Weber, L. A., Marshall, T. R., O'Reilly, J. X., & Hunt, L. T. (2022). Decision-making in dynamic, continuously evolving environments: Quantifying the flexibility of human choice (p. 2022.08.18.504278). bioRxiv https://doi.org/10.1101/2022.08.18.504278
Sambrook, T. D., & Goslin, J. (2015). A neural reward prediction error revealed by a meta-analysis of ERPs using great grand averages. Psychological Bulletin, 141(1), 213-235. https://doi.org/10.1037/bul0000006
Schultz, W., Dayan, P., & Montague, P. R. (1997). A neural substrate of prediction and reward. Science, 275(5306), 1593-1599. https://doi.org/10.1126/science.275.5306.1593
Schultz, W., Stauffer, W. R., & Lak, A. (2017). The phasic dopamine signal maturing: From reward via behavioural activation to formal economic utility. Current Opinion in Neurobiology, 43, 139-148. https://doi.org/10.1016/j.conb.2017.03.013
Smith, N. J., & Kutas, M. (2015a). Regression-based estimation of ERP waveforms: I. The rERP framework. Psychophysiology, 52(2), 157-168. https://doi.org/10.1111/psyp.12317
Smith, N. J., & Kutas, M. (2015b). Regression-based estimation of ERP waveforms: II. Nonlinear effects, overlap correction, and practical considerations. Psychophysiology, 52(2), 169-181. https://doi.org/10.1111/psyp.12320
Sutton, R. S., & Barto, A. G. (2018). Reinforcement learning: An introduction (2nd ed.). The MIT Press.
Ullsperger, M., Fischer, A. G., Nigbur, R., & Endrass, T. (2014). Neural mechanisms and temporal dynamics of performance monitoring. Trends in Cognitive Sciences, 18(5), 259-267. https://doi.org/10.1016/j.tics.2014.02.009
van Boxtel, G. J. M., & Böcker, K. B. E. (2004). Cortical measures of anticipation. Journal of Psychophysiology, 18(2-3), 61-76. https://doi.org/10.1027/0269-8803.18.23.61
Walsh, M. M., & Anderson, J. R. (2012). Learning from experience: Event-related potential correlates of reward processing, neural adaptation, and behavioral choice. Neuroscience & Biobehavioral Reviews, 36(8), 1870-1884. https://doi.org/10.1016/j.neubiorev.2012.05.008
Wang, Y., Toyoshima, O., Kunimatsu, J., Yamada, H., & Matsumoto, M. (2021). Tonic firing mode of midbrain dopamine neurons continuously tracks reward values changing moment-by-moment. eLife, 10, e63166. https://doi.org/10.7554/eLife.63166
Williams, C. C., Hassall, C. D., Trska, R., Holroyd, C. B., & Krigolson, O. E. (2017). When theory and biology differ: The relationship between reward prediction errors and expectancy. Biological Psychology, 129, 265-272. https://doi.org/10.1016/j.biopsycho.2017.09.007