Reinforcement learning establishes a minimal metacognitive process to monitor and control motor learning performance.


Journal

Nature communications
ISSN: 2041-1723
Titre abrégé: Nat Commun
Pays: England
ID NLM: 101528555

Informations de publication

Date de publication:
08 07 2023
Historique:
received: 06 09 2022
accepted: 16 06 2023
medline: 10 7 2023
pubmed: 9 7 2023
entrez: 8 7 2023
Statut: epublish

Résumé

Humans and animals develop learning-to-learn strategies throughout their lives to accelerate learning. One theory suggests that this is achieved by a metacognitive process of controlling and monitoring learning. Although such learning-to-learn is also observed in motor learning, the metacognitive aspect of learning regulation has not been considered in classical theories of motor learning. Here, we formulated a minimal mechanism of this process as reinforcement learning of motor learning properties, which regulates a policy for memory update in response to sensory prediction error while monitoring its performance. This theory was confirmed in human motor learning experiments, in which the subjective sense of learning-outcome association determined the direction of up- and down-regulation of both learning speed and memory retention. Thus, it provides a simple, unifying account for variations in learning speeds, where the reinforcement learning mechanism monitors and controls the motor learning process.

Identifiants

pubmed: 37422476
doi: 10.1038/s41467-023-39536-9
pii: 10.1038/s41467-023-39536-9
pmc: PMC10329706
doi:

Types de publication

Journal Article Research Support, N.I.H., Extramural Research Support, Non-U.S. Gov't

Langues

eng

Sous-ensembles de citation

IM

Pagination

3988

Subventions

Organisme : NINDS NIH HHS
ID : R21 NS120274
Pays : United States

Informations de copyright

© 2023. The Author(s).

Références

Neuron. 2022 Apr 20;110(8):1290-1303
pubmed: 35325616
Behav Res Methods. 2018 Aug;50(4):1614-1631
pubmed: 29949071
Nature. 2017 Apr 6;544(7648):96-100
pubmed: 28321129
Neuron. 2019 Aug 7;103(3):506-519.e4
pubmed: 31201123
Front Comput Neurosci. 2010 May 11;4:11
pubmed: 20485466
J Neurosci. 2006 Apr 5;26(14):3642-5
pubmed: 16597717
Trends Cogn Sci. 2007 Sep;11(9):387-92
pubmed: 17707682
Trends Cogn Sci. 2014 May;18(5):259-67
pubmed: 24656460
PLoS Biol. 2015 Dec 08;13(12):e1002312
pubmed: 26645916
J Exp Psychol. 1967 May;74(1):1-9
pubmed: 6032570
Curr Opin Neurobiol. 2008 Apr;18(2):185-96
pubmed: 18708140
Neural Netw. 2003 Jan;16(1):5-9
pubmed: 12576101
Neural Netw. 2022 Jan;145:80-89
pubmed: 34735893
PLoS Comput Biol. 2011 Mar;7(3):e1002012
pubmed: 21423711
Nat Neurosci. 2018 Jun;21(6):860-868
pubmed: 29760527
PLoS Comput Biol. 2022 Mar 23;18(3):e1010005
pubmed: 35320276
Nature. 2021 Dec;600(7889):489-493
pubmed: 34819674
Nat Rev Neurosci. 2018 Jun;19(6):338-350
pubmed: 29643480
J Neurosci Methods. 2009 Jul 30;181(2):199-211
pubmed: 19450621
Science. 2005 Nov 25;310(5752):1337-40
pubmed: 16311337
Neuron. 2019 Mar 20;101(6):1029-1041
pubmed: 30897355
Psychol Sci. 2008 Feb;19(2):103-8
pubmed: 18271855
Psychol Rev. 1949 Jan;56(1):51-65
pubmed: 18124807
Nat Commun. 2018 Mar 15;9(1):1099
pubmed: 29545572
Sci Rep. 2021 Jan 15;11(1):1627
pubmed: 33452363
Nature. 2004 Oct 14;431(7010):768-74
pubmed: 15483597
J Neurophysiol. 2021 Sep 1;126(3):934-945
pubmed: 34379553
Curr Opin Neurobiol. 2011 Jun;21(3):368-73
pubmed: 21531544
Neuron. 2012 Mar 22;73(6):1173-83
pubmed: 22445344
Nat Neurosci. 2015 Apr;18(4):597-602
pubmed: 25706473
Brain Res Cogn Brain Res. 2005 Apr;23(1):34-50
pubmed: 15795132
Nat Neurosci. 2011 Feb;14(2):154-62
pubmed: 21270784
J Neurosci. 2011 Jul 6;31(27):10050-9
pubmed: 21734297
J Neurosci. 2015 Oct 21;35(42):14386-96
pubmed: 26490874
J Neurosci. 2003 Oct 8;23(27):9032-45
pubmed: 14534237
J Neurosci. 2016 Mar 30;36(13):3839-47
pubmed: 27030768
J Neurosci. 2020 Apr 8;40(15):3075-3088
pubmed: 32029533
J Neurophysiol. 2015 Jan 15;113(2):633-46
pubmed: 25355957
J Neurosci. 2015 Jul 1;35(26):9568-79
pubmed: 26134640
Neural Netw. 2021 Oct;142:583-596
pubmed: 34352492
Science. 2007 Oct 26;318(5850):606-10
pubmed: 17962554
Science. 2004 Dec 10;306(5703):1940-3
pubmed: 15528409
Science. 2014 Sep 12;345(6202):1349-53
pubmed: 25123484
J Neurosci. 2005 Oct 26;25(43):9919-31
pubmed: 16251440
Elife. 2019 Apr 29;8:
pubmed: 31033439
J Cogn Neurosci. 2004 Jan-Feb;16(1):65-73
pubmed: 15006037
J Neurosci. 2012 Jan 11;32(2):551-62
pubmed: 22238090
Nat Neurosci. 2007 Jun;10(6):779-86
pubmed: 17496891
Nature. 2000 Oct 12;407(6805):742-7
pubmed: 11048720
Cogn Affect Behav Neurosci. 2008 Dec;8(4):429-53
pubmed: 19033240
J Mot Behav. 2010 Nov;42(6):371-9
pubmed: 21184355
Nat Neurosci. 2019 Jun;22(6):950-962
pubmed: 31036947
J Neurosci. 2012 Mar 21;32(12):4230-9
pubmed: 22442085
PLoS One. 2012;7(8):e43016
pubmed: 22916198
Elife. 2022 Feb 28;11:
pubmed: 35225229

Auteurs

Taisei Sugiyama (T)

Empowerment Informatics, University of Tsukuba, Tsukuba, Ibaraki, 305-8573, Japan.

Nicolas Schweighofer (N)

Biokinesiology and Physical Therapy, University of Southern California, Los Angeles, CA, 90089-9006, USA.

Jun Izawa (J)

Institute of Systems and Information Engineering, University of Tsukuba, Tsukuba, Ibaraki, 305-8573, Japan. izawa@emp.tsukuba.ac.jp.

Articles similaires

[Redispensing of expensive oral anticancer medicines: a practical application].

Lisanne N van Merendonk, Kübra Akgöl, Bastiaan Nuijen
1.00
Humans Antineoplastic Agents Administration, Oral Drug Costs Counterfeit Drugs

Smoking Cessation and Incident Cardiovascular Disease.

Jun Hwan Cho, Seung Yong Shin, Hoseob Kim et al.
1.00
Humans Male Smoking Cessation Cardiovascular Diseases Female
Humans United States Aged Cross-Sectional Studies Medicare Part C
1.00
Humans Yoga Low Back Pain Female Male

Classifications MeSH