Reinforcement learning establishes a minimal metacognitive process to monitor and control motor learning performance.
Journal
Nature communications
ISSN: 2041-1723
Titre abrégé: Nat Commun
Pays: England
ID NLM: 101528555
Informations de publication
Date de publication:
08 07 2023
08 07 2023
Historique:
received:
06
09
2022
accepted:
16
06
2023
medline:
10
7
2023
pubmed:
9
7
2023
entrez:
8
7
2023
Statut:
epublish
Résumé
Humans and animals develop learning-to-learn strategies throughout their lives to accelerate learning. One theory suggests that this is achieved by a metacognitive process of controlling and monitoring learning. Although such learning-to-learn is also observed in motor learning, the metacognitive aspect of learning regulation has not been considered in classical theories of motor learning. Here, we formulated a minimal mechanism of this process as reinforcement learning of motor learning properties, which regulates a policy for memory update in response to sensory prediction error while monitoring its performance. This theory was confirmed in human motor learning experiments, in which the subjective sense of learning-outcome association determined the direction of up- and down-regulation of both learning speed and memory retention. Thus, it provides a simple, unifying account for variations in learning speeds, where the reinforcement learning mechanism monitors and controls the motor learning process.
Identifiants
pubmed: 37422476
doi: 10.1038/s41467-023-39536-9
pii: 10.1038/s41467-023-39536-9
pmc: PMC10329706
doi:
Types de publication
Journal Article
Research Support, N.I.H., Extramural
Research Support, Non-U.S. Gov't
Langues
eng
Sous-ensembles de citation
IM
Pagination
3988Subventions
Organisme : NINDS NIH HHS
ID : R21 NS120274
Pays : United States
Informations de copyright
© 2023. The Author(s).
Références
Neuron. 2022 Apr 20;110(8):1290-1303
pubmed: 35325616
Behav Res Methods. 2018 Aug;50(4):1614-1631
pubmed: 29949071
Nature. 2017 Apr 6;544(7648):96-100
pubmed: 28321129
Neuron. 2019 Aug 7;103(3):506-519.e4
pubmed: 31201123
Front Comput Neurosci. 2010 May 11;4:11
pubmed: 20485466
J Neurosci. 2006 Apr 5;26(14):3642-5
pubmed: 16597717
Trends Cogn Sci. 2007 Sep;11(9):387-92
pubmed: 17707682
Trends Cogn Sci. 2014 May;18(5):259-67
pubmed: 24656460
PLoS Biol. 2015 Dec 08;13(12):e1002312
pubmed: 26645916
J Exp Psychol. 1967 May;74(1):1-9
pubmed: 6032570
Curr Opin Neurobiol. 2008 Apr;18(2):185-96
pubmed: 18708140
Neural Netw. 2003 Jan;16(1):5-9
pubmed: 12576101
Neural Netw. 2022 Jan;145:80-89
pubmed: 34735893
PLoS Comput Biol. 2011 Mar;7(3):e1002012
pubmed: 21423711
Nat Neurosci. 2018 Jun;21(6):860-868
pubmed: 29760527
PLoS Comput Biol. 2022 Mar 23;18(3):e1010005
pubmed: 35320276
Nature. 2021 Dec;600(7889):489-493
pubmed: 34819674
Nat Rev Neurosci. 2018 Jun;19(6):338-350
pubmed: 29643480
J Neurosci Methods. 2009 Jul 30;181(2):199-211
pubmed: 19450621
Science. 2005 Nov 25;310(5752):1337-40
pubmed: 16311337
Neuron. 2019 Mar 20;101(6):1029-1041
pubmed: 30897355
Psychol Sci. 2008 Feb;19(2):103-8
pubmed: 18271855
Psychol Rev. 1949 Jan;56(1):51-65
pubmed: 18124807
Nat Commun. 2018 Mar 15;9(1):1099
pubmed: 29545572
Sci Rep. 2021 Jan 15;11(1):1627
pubmed: 33452363
Nature. 2004 Oct 14;431(7010):768-74
pubmed: 15483597
J Neurophysiol. 2021 Sep 1;126(3):934-945
pubmed: 34379553
Curr Opin Neurobiol. 2011 Jun;21(3):368-73
pubmed: 21531544
Neuron. 2012 Mar 22;73(6):1173-83
pubmed: 22445344
Nat Neurosci. 2015 Apr;18(4):597-602
pubmed: 25706473
Brain Res Cogn Brain Res. 2005 Apr;23(1):34-50
pubmed: 15795132
Nat Neurosci. 2011 Feb;14(2):154-62
pubmed: 21270784
J Neurosci. 2011 Jul 6;31(27):10050-9
pubmed: 21734297
J Neurosci. 2015 Oct 21;35(42):14386-96
pubmed: 26490874
J Neurosci. 2003 Oct 8;23(27):9032-45
pubmed: 14534237
J Neurosci. 2016 Mar 30;36(13):3839-47
pubmed: 27030768
J Neurosci. 2020 Apr 8;40(15):3075-3088
pubmed: 32029533
J Neurophysiol. 2015 Jan 15;113(2):633-46
pubmed: 25355957
J Neurosci. 2015 Jul 1;35(26):9568-79
pubmed: 26134640
Neural Netw. 2021 Oct;142:583-596
pubmed: 34352492
Science. 2007 Oct 26;318(5850):606-10
pubmed: 17962554
Science. 2004 Dec 10;306(5703):1940-3
pubmed: 15528409
Science. 2014 Sep 12;345(6202):1349-53
pubmed: 25123484
J Neurosci. 2005 Oct 26;25(43):9919-31
pubmed: 16251440
Elife. 2019 Apr 29;8:
pubmed: 31033439
J Cogn Neurosci. 2004 Jan-Feb;16(1):65-73
pubmed: 15006037
J Neurosci. 2012 Jan 11;32(2):551-62
pubmed: 22238090
Nat Neurosci. 2007 Jun;10(6):779-86
pubmed: 17496891
Nature. 2000 Oct 12;407(6805):742-7
pubmed: 11048720
Cogn Affect Behav Neurosci. 2008 Dec;8(4):429-53
pubmed: 19033240
J Mot Behav. 2010 Nov;42(6):371-9
pubmed: 21184355
Nat Neurosci. 2019 Jun;22(6):950-962
pubmed: 31036947
J Neurosci. 2012 Mar 21;32(12):4230-9
pubmed: 22442085
PLoS One. 2012;7(8):e43016
pubmed: 22916198
Elife. 2022 Feb 28;11:
pubmed: 35225229