A One-Shot Shift from Explore to Exploit in Monkey Prefrontal Cortex.
attention
exploit
explore
frontal cortex
one-shot learning
primate
Journal
The Journal of neuroscience : the official journal of the Society for Neuroscience
ISSN: 1529-2401
Titre abrégé: J Neurosci
Pays: United States
ID NLM: 8102140
Informations de publication
Date de publication:
12 01 2022
12 01 2022
Historique:
received:
25
06
2021
revised:
19
08
2021
accepted:
07
10
2021
pubmed:
17
11
2021
medline:
15
2
2022
entrez:
16
11
2021
Statut:
ppublish
Résumé
Much animal learning is slow, with cumulative changes in behavior driven by reward prediction errors. When the abstract structure of a problem is known, however, both animals and formal learning models can rapidly attach new items to their roles within this structure, sometimes in a single trial. Frontal cortex is likely to play a key role in this process. To examine information seeking and use in a known problem structure, we trained monkeys in an explore/exploit task, requiring the animal first to test objects for their association with reward, then, once rewarded objects were found, to reselect them on further trials for further rewards. Many cells in the frontal cortex showed an explore/exploit preference aligned with one-shot learning in the monkeys' behavior: the population switched from an explore state to an exploit state after a single trial of learning but partially maintained the explore state if an error indicated that learning had failed. Binary switch from explore to exploit was not explained by continuous changes linked to expectancy or prediction error. Explore/exploit preferences were independent for two stages of the trial: object selection and receipt of feedback. Within an established task structure, frontal activity may control the separate processes of explore and exploit, switching in one trial between the two.
Identifiants
pubmed: 34782437
pii: JNEUROSCI.1338-21.2021
doi: 10.1523/JNEUROSCI.1338-21.2021
pmc: PMC8802942
doi:
Types de publication
Journal Article
Research Support, Non-U.S. Gov't
Langues
eng
Sous-ensembles de citation
IM
Pagination
276-287Subventions
Organisme : Medical Research Council
ID : MC_UU_00005/6
Pays : United Kingdom
Organisme : Medical Research Council
ID : MR/K005480/1
Pays : United Kingdom
Informations de copyright
Copyright © 2022 Achterberg et al.
Références
J Neurosci. 1996 Aug 15;16(16):5154-67
pubmed: 8756444
Nature. 2001 Jun 21;411(6840):953-6
pubmed: 11418860
J Exp Psychol Appl. 2014 Sep;20(3):215-31
pubmed: 25000309
J Neurosci. 2012 Mar 7;32(10):3499-515
pubmed: 22399773
J Neurosci. 2006 Mar 8;26(10):2745-56
pubmed: 16525054
Nat Neurosci. 2000 May;3(5):516-20
pubmed: 10769394
Neuron. 2010 May 13;66(3):438-48
pubmed: 20471356
J Neurosci. 2015 Feb 25;35(8):3478-84
pubmed: 25716847
Elife. 2019 Nov 18;8:
pubmed: 31738167
Science. 1997 Mar 14;275(5306):1593-9
pubmed: 9054347
Nat Hum Behav. 2018 Dec;2(12):915-924
pubmed: 30988442
J Neurosci. 2006 Nov 1;26(44):11313-23
pubmed: 17079659
J Neurophysiol. 1998 Jun;79(6):2919-40
pubmed: 9636098
PLoS Comput Biol. 2020 Apr 13;16(4):e1007720
pubmed: 32282795
J Neurosci. 2010 Nov 24;30(47):15801-10
pubmed: 21106819
J Neurosci. 2011 Aug 3;31(31):11110-7
pubmed: 21813672
Neuron. 2017 Mar 22;93(6):1504-1517.e4
pubmed: 28334612
Nat Commun. 2019 Jan 17;10(1):301
pubmed: 30655534
Neuroimage. 2013 Mar;68:162-72
pubmed: 23246992
J Neurosci. 2018 Aug 8;38(32):7020-7028
pubmed: 30089641
Nat Neurosci. 2013 Jan;16(1):105-10
pubmed: 23222911
Neuron. 2018 Oct 24;100(2):490-509
pubmed: 30359611
Exp Neurol. 1982 Sep;77(3):679-94
pubmed: 7117470
PLoS Comput Biol. 2016 Jun 10;12(6):e1004967
pubmed: 27286251
Nature. 2015 May 28;521(7553):436-44
pubmed: 26017442
Proc Natl Acad Sci U S A. 2008 Aug 19;105(33):11969-74
pubmed: 18689686
Nat Neurosci. 2018 Jun;21(6):860-868
pubmed: 29760527
Front Comput Neurosci. 2010 Oct 04;4:24
pubmed: 21048899
Nat Neurosci. 2004 Aug;7(8):887-93
pubmed: 15235607
J Exp Psychol Gen. 2014 Dec;143(6):2074-81
pubmed: 25347535
Behav Brain Sci. 2017 Jan;40:e253
pubmed: 27881212
Nature. 2020 Sep;585(7825):357-362
pubmed: 32939066
Cereb Cortex. 2006 Dec;16(12):1679-89
pubmed: 16436686
Psychol Rev. 1949 Jan;56(1):51-65
pubmed: 18124807
Cereb Cortex. 2015 Sep;25(9):3197-218
pubmed: 24904073
Nat Methods. 2020 Mar;17(3):261-272
pubmed: 32015543
Cereb Cortex. 2018 Nov 1;28(11):3816-3828
pubmed: 29040434
Neuron. 2020 Jun 17;106(6):1044-1054.e4
pubmed: 32315603
Science. 2003 Jul 11;301(5630):229-32
pubmed: 12855813
Neuron. 2006 May 18;50(4):631-41
pubmed: 16701212
Science. 1971 Aug 13;173(3997):652-4
pubmed: 4998337
J Neurophysiol. 1989 Feb;61(2):331-49
pubmed: 2918358
Nature. 2013 May 30;497(7451):585-90
pubmed: 23685452
Cereb Cortex. 2020 Jun 30;30(8):4336-4345
pubmed: 32239139
Prog Neurobiol. 2013 Apr;103:214-22
pubmed: 23438479
Nat Neurosci. 1998 May;1(1):80-4
pubmed: 10195114
Neuron. 2020 Aug 19;107(4):603-616
pubmed: 32663439
Neuron. 2008 Jan 24;57(2):314-25
pubmed: 18215627
Nat Neurosci. 2011 Oct 30;14(12):1581-9
pubmed: 22037498
Cereb Cortex. 2020 Mar 14;30(3):1779-1796
pubmed: 31690931
Proc Natl Acad Sci U S A. 2009 Jun 9;106(23):9530-3
pubmed: 19470493
Neural Netw. 2015 Jan;61:85-117
pubmed: 25462637
Proc Natl Acad Sci U S A. 2018 Jul 24;115(30):E7202-E7211
pubmed: 29991597