One Size Does Not Fit All: Idiographic Computational Models Reveal Individual Differences in Learning and Meta-Learning Strategies.
ACT‐R
Declarative memory
Individual differences
Learning
Reinforcement learning
Working memory
Journal
Topics in cognitive science
ISSN: 1756-8765
Titre abrégé: Top Cogn Sci
Pays: United States
ID NLM: 101506764
Informations de publication
Date de publication:
03 Apr 2024
03 Apr 2024
Historique:
revised:
07
03
2024
received:
20
07
2022
accepted:
18
03
2024
medline:
3
4
2024
pubmed:
3
4
2024
entrez:
3
4
2024
Statut:
aheadofprint
Résumé
Complex skill learning depends on the joint contribution of multiple interacting systems: working memory (WM), declarative long-term memory (LTM) and reinforcement learning (RL). The present study aims to understand individual differences in the relative contributions of these systems during learning. We built four idiographic, ACT-R models of performance on the stimulus-response learning, Reinforcement Learning Working Memory task. The task consisted of short 3-image, and long 6-image, feedback-based learning blocks. A no-feedback test phase was administered after learning, with an interfering task inserted between learning and test. Our four models included two single-mechanism RL and LTM models, and two integrated RL-LTM models: (a) RL-based meta-learning, which selects RL or LTM to learn based on recent success, and (b) a parameterized RL-LTM selection model at fixed proportions independent of learning success. Each model was the best fit for some proportion of our learners (LTM: 68.7%, RL: 4.8%, Meta-RL: 13.25%, bias-RL:13.25% of participants), suggesting fundamental differences in the way individuals deploy basic learning mechanisms, even for a simple stimulus-response task. Finally, long-term declarative memory seems to be the preferred learning strategy for this task regardless of block length (3- vs 6-image blocks), as determined by the large number of subjects whose learning characteristics were best captured by the LTM only model, and a preference for LTM over RL in both of our integrated-models, owing to the strength of our idiographic approach.
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Subventions
Organisme : Office of Naval Research
ID : N00014-20-1-2393
Informations de copyright
© 2024 Cognitive Science Society LLC.
Références
Anderson, J. R. (2000). Learning and memory: An integrated approach. John Wiley & Sons Inc.
Anderson, J. R. (2007). How can the human mind occur in the physical universe? Oxford University Press.
Anderson, J. R. (1982). Acquisition of cognitive skill. Psychological review, 89(4), 369.
Antzoulatos, E. G., & Miller, E. K. (2014). Increases in functional connectivity between prefrontal cortex and striatum during category learning. Neuron, 83(1), 216–225.
Ceballos, J. M., Stocco, A., & Prat, C. S. (2020). The role of basal ganglia reinforcement learning in lexical ambiguity resolution. Topics in Cognitive Science, 12(1), 402–416.
Cetron, J. S., Connolly, A. C., Diamond, S. G., May, V. V., Haxby, J. V., & Kraemer, D. J. (2020). Using the force: STEM knowledge and experience construct shared neural representations of engineering concepts. NPJ Science of Learning, 5(1), 1–10.
Collins, A. G. (2018). The tortoise and the hare: Interactions between reinforcement learning and working memory. Journal of Cognitive Neuroscience, 30(10), 1422–1432.
Collins, A. G., & Frank, M. J. (2012). How much of reinforcement learning is working memory, not reinforcement learning? A behavioral, computational, and neurogenetic analysis. European Journal of Neuroscience, 35(7), 1024–1035.
Daily, L. Z., Lovett, M. C., & Reder, L. M. (2001). Modeling individual differences in working memory performance: A source activation account. Cognitive Science, 25(3), 315–353.
Daw, N. D. (2011). Trial‐by‐trial data analysis using computational models. In Decision making, affect, and learning: Attention and performance XXIII.
DeCaro, M. S., Thomas, R. D., & Beilock, S. L. (2008). Individual differences in category learning: Sometimes less working memory capacity is better than more. Cognition, 107(1), 284–294.
Engle, R. W. (2002). Working memory capacity as executive attention. Current directions in psychological science, 11(1), 19–23.
Fisher, C. R., Walsh, M. M., Blaha, L. M., Gunzelmann, G., & Veksler, B. (2016). Efficient parameter estimation of cognitive models for real‐time performance monitoring and adaptive interfaces. In Proceedings of the 14th International Conference on Cognitive Modeling (ICCM 2016). University Park, PA.
Frank, M. J., Moustafa, A. A., Haughey, H. M., Curran, T., & Hutchison, K. E. (2007). Genetic triple dissociation reveals multiple roles for dopamine in reinforcement learning. Proceedings of the National Academy of Sciences, 104(41), 16311–16316.
Gibson, E., Futrell, R., Jara‐Ettinger, J., Mahowald, K., Bergen, L., Ratnasingam, S., Gibson, M., Piantadosi, S. T., & Conway, B. R. (2017). Color naming across languages reflects color use. Proceedings of the National Academy of Sciences, 114(40), 10785–10790.
Gluck, M. A. (2002). How do people solve the “weather prediction” task? Individual variability in strategies for probabilistic category learning. Learning & Memory, 9(6), 408–418.
Gonzalez, C., Lerch, J. F., & Lebiere, C. (2003). Instance‐based learning in dynamic decision making. Cognitive Science, 27(4), 591–635.
Hedge, C., Powell, G., & Sumner, P. (2018). The reliability paradox: Why robust cognitive tasks do not produce reliable individual differences. Behavior research methods, 50, 1166–1186.
Hikosaka, O., Yamamoto, S., Yasuda, M., & Kim, H. F. (2013). Why skill matters. Trends in Cognitive Sciences, 17(9), 434–441.
Just, M. A., & Carpenter, P. A. (1992). A capacity theory of comprehension: Individual differences in working memory. Psychological Review, 99(1), 122.
Kalra, P. B., Gabrieli, J. D., & Finn, A. S. (2019). Evidence of stable individual differences in implicit learning. Cognition, 190, 199–211
Kane, M. J., Bleckley, M. K., Conway, A. R., & Engle, R. W. (2001). A controlled‐attention view of working‐memory capacity. Journal of Experimental Psychology: General, 130(2), 169.
Kass, R. E., & Raftery, A. E. (1995). Bayes factors. Journal of the American Statistical Association, 90(430), 773–795.
Knowlton, B. J., & Squire, L. R. (1996). Artificial grammar learning depends on implicit acquisition of both abstract and exemplar‐specific information. Journal of Experimental Psychology: Learning, Memory, and Cognition, 22(1), 169.
Laird, J. E., Lebiere, C., & Rosenbloom, P. S. (2017). A standard model of the mind: Toward a common computational framework across artificial intelligence, cognitive science, neuroscience, and robotics. AI Magazine, 38(4), 13–26.
Lovett, M. C., Daily, L. Z., & Reder, L. M. (2000). A source activation theory of working memory: Cross‐task prediction of performance in ACT‐R. Cognitive Systems Research, 1(2), 99–118.
Lupyan, G., Rakison, D. H., & McClelland, J. L. (2007). Language is not just for talking: Redundant labels facilitate learning of novel categories. Psychological Science, 18(12), 1077–1083.
McDonald, R. J., & Hong, N. S. (2013). How does a specific learning and memory system in the mammalian brain gain control of behavior? Hippocampus, 23(11), 1084–1102.
Miller, E. K., Lundqvist, M., & Bastos, A. M. (2018). Working memory 2.0. Neuron, 100(2), 463–475.
Niv, Y. (2009). Reinforcement learning in the brain. Journal of Mathematical Psychology, 53(3), 139–154
Poldrack, R. A., Clark, J., Paré‐Blagoev, E. J., Shohamy, D., Creso Moyano, J., Myers, C., & Gluck, M. A. (2001). Interactive memory systems in the human brain. Nature, 414(6863), 546–550.
Poldrack, R. A., & Packard, M. G. (2003). Competition among multiple memory systems: Converging evidence from animal and human brain studies. Neuropsychologia, 41(2003), 245–2517.
Puig, M. V., Antzoulatos, E. G., & Miller, E. K. (2014). Prefrontal dopamine in associative learning and memory. Neuroscience, 282(2014), 217–229.
Schneider, W., & Chein, J. M. (2003). Controlled & automatic processing: behavior, theory, and biological mechanisms. Cognitive science, 27(3), 525–559.
Schwarz, G. (1978). Estimating the Dimension of a Model. The Annals of Statistics, 6(2), 461–464.
Sense, F., Behrens, F., Meijer, R. R., & van Rijn, H. (2016). An individual's rate of forgetting is stable over time but differs across materials. Topics in cognitive science, 8(1), 305–321.
Squire, L. R. (2004). Memory systems of the brain: A brief history and current perspective. Neurobiology of Learning and Memory, 82(3), 171–177.
Stewart, N., Chater, N., & Brown, G. D. (2006). Decision by sampling. Cognitive Psychology, 53(1), 1–26.
Stocco, A., Lebiere, C., & Anderson, J. R. (2010). Conditional routing of information to the cortex: A model of the basal ganglia's role in cognitive coordination. Psychological Review, 117(2), 541.
Stocco, A., Murray, N. L., Yamasaki, B. L., Renno, T. J., Nguyen, J., & Prat, C. S. (2017). Individual differences in the Simon effect are underpinned by differences in the competitive dynamics in the basal ganglia: An experimental verification and a computational model. Cognition, 164, 31–45.
Ullman, M. T. (2001). A neurocognitive perspective on language: The declarative/procedural model. Nature reviews neuroscience, 2(10), 717–726.
Xu, Y., & Stocco, A. (2021). Recovering reliable idiographic biological parameters from noisy behavioral data: The case of basal ganglia indices in the probabilistic selection task. Computational Brain & Behavior, 4, 318–334.
Zeithamova, D., & Maddox, W. T. (2006). Dual‐task interference in perceptual category learning. Memory & Cognition, 34(2), 387–398.