The Role of Basal Ganglia Reinforcement Learning in Lexical Ambiguity Resolution.
ACT-R
Ambiguity resolution
Basal ganglia
Cognitive modeling
Dopamine
Language
Lexical selection
Priming
Reinforcement learning
Semantics
Journal
Topics in cognitive science
ISSN: 1756-8765
Titre abrégé: Top Cogn Sci
Pays: United States
ID NLM: 101506764
Informations de publication
Date de publication:
01 2020
01 2020
Historique:
received:
27
09
2019
accepted:
08
11
2019
revised:
15
12
2019
entrez:
6
2
2020
pubmed:
6
2
2020
medline:
18
5
2021
Statut:
ppublish
Résumé
The current study aimed to elucidate the contributions of the subcortical basal ganglia to human language by adopting the view that these structures engage in a basic neurocomputation that may account for its involvement across a wide range of linguistic phenomena. Specifically, we tested the hypothesis that basal ganglia reinforcement learning (RL) mechanisms may account for variability in semantic selection processes necessary for ambiguity resolution. To test this, we used a biased homograph lexical ambiguity priming task that allowed us to measure automatic processes for resolving ambiguity toward high-frequency word meanings. Individual differences in task performance were then related to indices of basal ganglia RL, which were used to group subjects into three learning styles: (a) Choosers who learn by seeking high reward probability stimuli; (b) Avoiders, who learn by avoiding low reward probability stimuli; and (c) Balanced participants, whose learning reflects equal contributions of choose and avoid processes. The results suggest that balanced individuals had significantly lower access to subordinate, or low-frequency, homograph word meanings. Choosers and Avoiders, on the other hand, had higher access to the subordinate word meaning even after a long delay between prime and target. Experimental findings were then tested using an ACT-R computational model of RL that learns from both positive and negative feedback. Results from the computational model simulations confirm and extend the pattern of behavioral findings, providing an RL account of individual differences in lexical ambiguity resolution.
Types de publication
Journal Article
Research Support, Non-U.S. Gov't
Langues
eng
Sous-ensembles de citation
IM
Pagination
402-416Subventions
Organisme : National Science Foundation Graduate Research Fellowship
ID : DGE-1256082
Pays : International
Organisme : Office of Naval Research
ID : N00014-17-1-2607
Pays : International
Informations de copyright
© 2020 Cognitive Science Society, Inc.
Références
Alexander, G. E., Crutcher, M. D., & DeLong, M. R. (1991). Basal ganglia-thalamocortical circuits: Parallel substrates for motor, oculomotor, “prefrontal” and “limbic” functions. In H. B. M. Uylings, C. G. Van Eden, J. P. C. De Bruin, M. A. Corner, & M. G. P. Feenstra (Eds.),Progress in brain research (Vol. 85, pp. 119-146). Amsterdam: Elsevier.
Anderson, J. R. (2007). Oxford series on cognitive models and architectures. How can the human mind occur in the physical universe? New York: Oxford University Press. Https://doi.org/10.1093/acprof:oso/9780195324259.001.0001
Anderson, J. R., Bothell, D., Byrne, M. D., Douglass, S., Lebiere, C., & Qin, Y. (2004). An integrated theory of the mind. Psychological Review, 111(4), 1036-1060.
Anderson, J. R., Fincham, J. M., Qin, Y., & Stocco, A. (2008). A central circuit of the mind. Trends in Cognitive Sciences, 12(4), 136-143.
Bates, D., Mächler, M., Bolker, B., & Walker, S. (2015). Fitting linear mixed-effects models using lme4. arXiv preprint, https://www.jstatsoft.org/article/view/v067i01
Booth, J. R., Wood, L., Lu, D., Houk, J. C., & Bitan, T. (2007). The role of the basal ganglia and cerebellum in language processing. Brain Research, 1133, 136-144.
Collins, A. M., & Loftus, E. F. (1975). A spreading-activation theory of semantic processing. Psychological Review, 82(6), 407.
Copland, D. A., Chenery, H. J., & Murdoch, B. E. (2001). Discourse priming of homophones in individuals with dominant nonthalamic subcortical lesions, cortical lesions and Parkinson's disease. Journal of Clinical and Experimental Neuropsychology, 23(4), 538-556.
Crosson, B. (1985). Subcortical functions in language: A working model. Brain and Language, 25(2), 257-292.
Duffy, S. A., Morris, R. K., & Rayner, K. (1988). Lexical ambiguity and fixation times in reading. Journal of Memory and Language, 27(4), 429-446.
Frank, M. J., & Hutchison, K. (2009). Genetic contributions to avoidance-based decisions: Striatal D2 receptor polymorphisms. Neuroscience, 164(1), 131-140.
Frank, M. J., Moustafa, A. A., Haughey, H. M., Curran, T., & Hutchison, K. E. (2007). Genetic triple dissociation reveals multiple roles for dopamine in reinforcement learning. Proceedings of the National Academy of Sciences, 104(41), 16311-16316.
Frank, M. J., Seeberger, L. C., & O'reilly, R. C. (2004). By carrot or by stick: Cognitive reinforcement learning in parkinsonism. Science, 306(5703), 1940-1943.
Graybiel, A. M. (1995). Building action repertoires: Memory and learning functions of the basal ganglia. Current Opinion in Neurobiology, 5(6), 733-741.
Gurney, K., Prescott, T. J., & Redgrave, P. (2001). A computational model of action selection in the basal ganglia. I. A new functional anatomy. Biological Cybernetics, 84(6), 401-410.
Hasselt, H. V. (2010). Double Q-learning. In J. D. Lafferty, C. K. I. Williams, J. Shawe-Taylor, R. S. Zemel, & A. Culotta (Eds.), Advances in neural information processing systems (pp. 2613-2621). Vancouver, Canada: Advances in Neural Information Processing Systems 23.
Hazy, T. E., Frank, M. J., & O'Reilly, R. C. (2007). Towards an executive without a homunculus: computational models of the prefrontal cortex/basal ganglia system. Philosophical Transactions of the Royal Society B: Biological Sciences, 362(1485), 1601-1613.
Hernandez, A. E., Claussenius-Kalman, H. L., Ronderos, J., Castilla-Earls, A. P., Sun, L., Weiss, S. D., & Young, D. R. (2019). Neuroemergentism: A framework for studying cognition and the brain. Journal of Neurolinguistics, 49, 214-223.
Joel, D., Niv, Y., & Ruppin, E. (2002). Actor-critic models of the basal ganglia: New anatomical and computational perspectives. Neural Networks, 15(4-6), 535-547.
Ketteler, D., Kastrau, F., Vohn, R., & Huber, W. (2008). The subcortical role of language processing. High level linguistic features such as ambiguity-resolution and the human brain; An fMRI study. NeuroImage, 39(4), 2002-2009.
Kotz, S. A., Schwartze, M., & Schmidt-Kassow, M. (2009). Non-motor basal ganglia functions: A review and proposal for a model of sensory predictability in auditory language perception. Cortex, 45(8), 982-990.
Kristensen, M., & Hansen, T. (2004). Statistical analyses of repeated measures in physiological research: A tutorial. Advances in Physiology Education, 28(1), 2-14.
Lieberman, P. (2001). Human language and our reptilian brain: The subcortical bases of speech, syntax, and thought. Perspectives in Biology and Medicine, 44(1), 32-51.
Mason, R. A., & Just, M. A. (2007). Lexical ambiguity in sentence comprehension. Brain Research, 1146, 115-127.
McNab, F., & Klingberg, T. (2008). Prefrontal cortex and basal ganglia control access to working memory. Nature Neuroscience, 11(1), 103-107.
Mink, J. W. (1996). The basal ganglia: Focused selection and inhibition of competing motor programs. Progress in Neurobiology, 50(4), 381-425.
O'Reilly, R. C., & Frank, M. J. (2006). Making working memory work: A computational model of learning in the prefrontal cortex and basal ganglia. Neural Computation, 18(2), 283-328.
Randolph, C. (1991). Implicit, explicit, and semantic memory functions in Alzheimer's disease and Huntington's disease. Journal of Clinical and Experimental Neuropsychology, 13(4), 479-494.
Rice, P. J., & Stocco, A. (2017). Basal ganglia-inspired functional constraints improve the robustness of Q-value estimates in model-free reinforcement learning. In M. K. van Vugt, A. P. Banks, & W. G. Kennedy (Eds.), Proceedings of the 15th International Conference on Cognitive Modeling (pp. 67-72). Coventry, UK: University of Warwick.
Schnur, T. T., Schwartz, M. F., Kimberg, D. Y., Hirshorn, E., Coslett, H. B., & Thompson-Schill, S. L. (2009). Localizing interference during naming: Convergent neuroimaging and neuropsychological evidence for the function of Broca's area. Proceedings of the National Academy of Sciences of the United States of America, 106(1), 322-327.
Seo, R., Stocco, A., & Prat, C. S. (2018). The bilingual language network: Differential involvement of anterior cingulate, basal ganglia and prefrontal cortex in preparation, monitoring, and execution. NeuroImage, 174, 44-56.
Stanovich, K. E., & West, R. F. (1989). Exposure to print and orthographic processing. Reading Research Quarterly, 24(4), 402-433.
Stocco, A. (2018). A biologically plausible action selection system for cognitive architectures: Implications of basal ganglia anatomy for learning and decision-making models. Cognitive Science, 42(2), 457-490.
Stocco, A., Lebiere, C., & Anderson, J. R. (2010). Conditional routing of information to the cortex: A model of the basal ganglia's role in cognitive coordination. Psychological Review, 117(2), 541.
Stocco, A., Murray, N. L., Yamasaki, B. L., Renno, T. J., Nguyen, J., & Prat, C. S. (2017). Individual differences in the Simon effect are underpinned by differences in the competitive dynamics in the basal ganglia: An experimental verification and a computational model. Cognition, 164, 31-45.
Sutton, R. S. (1988). Learning to predict by the methods of temporal differences. Machine Learning, 3(1), 9-44.
Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning: An introduction. Cambridge, MA: MIT Press.
Tizhoosh, H. R. (2005). Opposition-based learning: A new scheme for machine intelligence. In M. Mohammadian (Ed.), International conference on Computational Intelligence for Modelling, Control and Automation and International Conference on Intelligent Agents, Web Technologies and Internet Commerce (CIMCA-IAWTIC'06) (Vol. 1, pp. 695-701). Vienna, Austria: IEEE.
Tremblay, P., & Dick, A. S. (2016). Broca and Wernicke are dead, or moving past the classic model of language neurobiology. Brain and Language, 162, 60-71.
Twilley, L. C., Dixon, P., Taylor, D., & Clark, K. (1994). University of Alberta norms of relative meaning frequency for 566 homographs. Memory & Cognition, 22(1), 111-126.
Vitello, S., & Rodd, J. M. (2015). Resolving semantic ambiguities in sentences: Cognitive processes and brain mechanisms. Language and Linguistics Compass, 9(10), 391-405.
Watkins, C. J., & Dayan, P. (1992). Q-learning. Machine Learning, 8(3-4), 279-292.