Probabilistic value learning in medial temporal lobe amnesia.
amnesia
hippocampus
medial temporal lobes
reinforcement learning
reward learning
Journal
Hippocampus
ISSN: 1098-1063
Titre abrégé: Hippocampus
Pays: United States
ID NLM: 9108167
Informations de publication
Date de publication:
05 2021
05 2021
Historique:
revised:
04
02
2021
received:
23
10
2020
accepted:
06
02
2021
pubmed:
28
2
2021
medline:
25
2
2022
entrez:
27
2
2021
Statut:
ppublish
Résumé
A prevailing view in cognitive neuroscience suggests that different forms of learning are mediated by dissociable memory systems, with a mesolimbic (i.e., midbrain and basal ganglia) system supporting incremental trial-and-error reinforcement learning and a hippocampal-based system supporting episodic memory. Yet, growing evidence suggests that the hippocampus may also be important for trial-and-error learning, particularly value or reward-based learning. In the present report, we use a lesion-based neuropsychological approach to clarify hippocampal contributions to such learning. Six amnesic patients with medial temporal lobe damage and a group of healthy controls were administered a simple value-based learning task involving probabilistic trial-and-error acquisition of stimulus-response-outcome (reward or none) contingencies modeled after Li et al. (Proceedings of the National Academy of Sciences , 2011, 108 (1), 55-60). As predicted, patients were significantly impaired on the task, demonstrating reduced learning of the contingencies. Our results provide further supportive evidence that the hippocampus' role in cognition extends beyond episodic memory tasks and call for further refinement of theoretical models of hippocampal functioning.
Types de publication
Journal Article
Research Support, Non-U.S. Gov't
Research Support, U.S. Gov't, Non-P.H.S.
Langues
eng
Sous-ensembles de citation
IM
Pagination
461-468Informations de copyright
Published 2021. This article is a U.S. Government work and is in the public domain in the USA.
Références
Adcock, R. A., Thangavel, A., Whitfield-Gabrieli, S., Knutson, B., & Gabrieli, J. D. (2006). Reward-motivated learning: Mesolimbic activation precedes memory formation. Neuron, 50(3), 507-517. https://doi.org/10.1016/j.neuron.2006.03.036
Ballard, I., Wagner, A. D., & McClure, S. M. (2018). Hippocampal pattern separation supports reinforcement learning. Nature Communications, 10, 1073. https://doi.org/10.1038/s41467-019-08998-1
Delgado, M. R., Nystrom, L. E., Fissell, C., Noll, D. C., & Fiez, J. A. (2000). Tracking the hemodynamic responses to reward and punishment in the striatum. Journal of Neurophysiology, 84(6), 3072-3077. https://doi.org/10.1152/jn.2000.84.6.3072
Dickerson, K. C., Li, J., & Delgado, M. R. (2011). Parallel contributions of distinct human memory systems during probabilistic learning. NeuroImage, 55(1), 266-276. https://doi.org/10.1016/j.neuroimage.2010.10.080
Duncan, K., Doll, B. B., Daw, N. D., & Shohamy, D. (2018). More than the sum of its parts: A role for the hippocampus in configural reinforcement learning. Neuron, 98(3), 645-657.e6. doi:https://doi.org/10.1016/j.neuron.2018.03.042
Floresco, S. B., Todd, C. L., & Grace, A. A. (2001). Glutamatergic afferents from the hippocampus to the nucleus accumbens regulate activity of ventral tegmental area dopamine neurons. Journal of Neuroscience, 21(13), 4915-4922. https://doi.org/10.1523/jneurosci.21-13-04915.2001
Foerde, K., Race, E., Verfaellie, M., & Shohamy, D. (2013). A role for the medial temporal lobe in feedback-driven learning: Evidence from amnesia. Journal of Neuroscience, 33(13), 5698-5704. https://doi.org/10.1523/jneurosci.5217-12.2013
Foreman-Mackey, D., Hogg, D. W., Lang, D., & Goodman, J. (2013). emcee: The MCMC hammer. Publications of the Astronomical Society of the Pacific, 125(925), 306-312. https://doi.org/10.1086/670067
Gasbarri, A., Packard, M. G., Campana, E., & Pacitti, C. (1994). Anterograde and retrograde tracing of projections from the ventral tegmental area to the hippocampal formation in the rat. Brain Research Bulletin, 33(4), 445-452. https://doi.org/10.1016/0361-9230(94)90288-7
Goodman, J., & Weare, J. (2010). Ensemble samplers with affine invariance. Communications in Applied Mathematics and Computational Science, 5(1), 65-80. https://doi.org/10.2140/camcos.2010.5.65
Grinsted, A. (2015). Grinsted/gwmcmc [Matlab]. https://github.com/grinsted/gwmcmc
Groenewegen, H. J., der Zee, E. V.-V., te Kortschot, A., & Witter, M. P. (1987). Organization of the projections from the subiculum to the ventral striatum in the rat. A study using anterograde transport of Phaseolus vulgaris leucoagglutinin. Neuroscience, 23(1), 103-120. https://doi.org/10.1016/0306-4522(87)90275-2
Hopkins, R. O., Myers, C. E., Shohamy, D., Grossman, S., & Gluck, M. (2004). Impaired probabilistic category learning in hypoxic subjects with hippocampal damage. Neuropsychologia, 42(4), 524-535. https://doi.org/10.1016/j.neuropsychologia.2003.09.005
Kahn, I., & Shohamy, D. (2013). Intrinsic connectivity between the hippocampus, nucleus accumbens, and ventral tegmental area in humans. Hippocampus, 23(3), 187-192. https://doi.org/10.1002/hipo.22077
Kan, I. P., Giovanello, K. S., Schnyer, D. M., Makris, N., & Verfaellie, M. (2007). Role of the medial temporal lobes in relational memory: Neuropsychological evidence from a cued recognition paradigm. Neuropsychologia, 45(11), 2589-2597. https://doi.org/10.1016/j.neuropsychologia.2007.03.006
Knowlton, B. J., Mangels, J. A., & Squire, L. R. (1996). A neostriatal habit learning system in humans. Science, 273(5280), 1399-1402. https://doi.org/10.1126/science.273.5280.1399
Knowlton, B. J., Squire, L. R., & Gluck, M. A. (1994). Probabilistic classification learning in amnesia. Learning & Memory, 1(2), 106-120. doi:https://doi.org/10.1101/lm.1.2.106
Lee, H., Ghim, J. W., Kim, H., Lee, D., & Jung, M. (2012). Hippocampal neural correlates for values of experienced events. Journal of Neuroscience, 32(43), 15053-15065. https://doi.org/10.1523/jneurosci.2806-12.2012
Leutgeb, J. K., Leutgeb, S., Moser, M.-B., & Moser, E. I. (2007). Pattern separation in the dentate gyrus and CA3 of the hippocampus. Science (New York, N.Y.), 315(5814), 961-966. doi:https://doi.org/10.1126/science.1135801
Li, J., Delgado, M. R., & Phelps, E. A. (2011). How instructed knowledge modulates the neural systems of reward learning. Proceedings of the National Academy of Sciences, 108(1), 55-60. https://doi.org/10.1073/pnas.1014938108
Lodge, D. J., & Grace, A. A. (2006). The hippocampus modulates dopamine neuro nresponsivity by regulating the intensity of phasic neuron activation. Neuropsychopharmacology, 31(7), 1356-1361. https://doi.org/10.1038/sj.npp.1300963
Madan, C. R., Fujiwara, E., Gerson, B. C., & Caplan, J. B. (2012). High reward makes items easier to remember, but harder to bind to a new temporal context. Frontiers in Integrative Neuroscience, 6, 61. https://doi.org/10.3389/fnint.2012.00061
Murty, V. P., & Adcock, R. A. (2014). Enriched encoding: Reward motivation organizes cortical networks for hippocampal detection of unexpected events. Cerebral Cortex, 24(8), 2160-2168. https://doi.org/10.1093/cercor/bht063
Murty V. P., Dickerson K. C. (2017). Motivational influences on memory, Advances in Motivation and Achievement (pg. 203-227). Bingley, UK: Emerald Group Publishing Limited.
Palombo, D. J., Hayes, S. M., Reid, A. G., & Verfaellie, M. (2019). Hippocampal contributions to value-based learning: Converging evidence from fMRI and amnesia. Cognitive, Affective, & Behavioral Neuroscience, 19(3), 523-536. https://doi.org/10.3758/s13415-018-00687-8
Rutledge, R. B., Lazzaro, S. C., Lau, B., Myers, C. E., Gluck, M. A., & Glimcher, P. W. (2009). Dopaminergic drugs modulate learning rates and perseveration in Parkinson's patients in a dynamic foraging task. Journal of Neuroscience, 29(48), 15104-15114. https://doi.org/10.1523/jneurosci.3524-09.2009
Schonberg, T., O'Doherty, J. P., Joel, D., Inzelberg, R., Segev, Y., & Daw, N. D. (2010). Selective impairment of prediction error signaling in human dorsolateral but not ventral striatum in Parkinson's disease patients: Evidence from a model-based fMRI study. NeuroImage, 49(1), 772-781. https://doi.org/10.1016/j.neuroimage.2009.08.011
Seib, D. R., Espinueva, D. F., Floresco, S. B., & Snyder, J. S. (2020). A role for neurogenesis in probabilistic reward learning. Behavioral Neuroscience, 134(4), 283-295. https://doi.org/10.1037/bne0000370
Shohamy, D., Myers, C. E., Kalanithi, J., & Gluck, M. A. (2008). Basal ganglia and dopamine contributions to probabilistic category learning. Neuroscience and Biobehavioral Reviews, 32(2), 219-236. https://doi.org/10.1016/j.neubiorev.2007.07.008
Squire, L. R. (1987). Memory and brain. New York, NY: Oxford University Press.
Sutton, R. S., & Barto, A. G. (2018). Reinforcement learning: An introduction. London, UK: MIT Press.
Vanni-Mercier, G., Mauguière, F., Isnard, J., & Dreher, J.-C. (2009). The hippocampus codes the uncertainty of cue-outcome associations: An intracranial electrophysiological study in humans. Journal of Neuroscience, 29(16), 5287-5294. https://doi.org/10.1523/jneurosci.5298-08.2009
Wolosin, S. M., Zeithamova, D., & Preston, A. R. (2012). Reward modulation of hippocampal subfield activation during successful associative encoding and retrieval. Journal of Cognitive Neuroscience, 24(7), 1532-1547. https://doi.org/10.1162/jocn_a_00237