A causal role of the right dorsolateral prefrontal cortex in random exploration.
Journal
Scientific reports
ISSN: 2045-2322
Titre abrégé: Sci Rep
Pays: England
ID NLM: 101563288
Informations de publication
Date de publication:
22 10 2024
22 10 2024
Historique:
received:
01
05
2024
accepted:
09
10
2024
medline:
22
10
2024
pubmed:
22
10
2024
entrez:
21
10
2024
Statut:
epublish
Résumé
Decision to explore new options with uncertain outcomes or exploit familiar options with known outcomes is a fundamental challenge that the brain faces in almost all real-life decisions. Previous studies have shown that humans use two main explorative strategies to negotiate this explore-exploit tradeoff. Exploring for the sake of information is called directed exploration, and exploration driven by behavioral variability is known as random exploration. While previous neuroimaging studies have shown different neural correlates for these explorative strategies, including right frontopolar cortex (FPC), right dorsolateral prefrontal cortex (DLPFC), and dorsal anterior cingulate cortex (dACC), there is still a lack of causal evidence for most of these brain regions. Here, we focused on the right DLPFC, which was previously supported to be involved in exploration. Using the continuous theta burst stimulation (cTBS) and Horizon task on twenty-five healthy right-handed adult participants, we showed that inhibiting rDLPFC did not change directed exploration but selectively reduced random exploration, by increasing reward sensitivity over the average reward of each option. This suggests a causal role for rDLPFC in random exploration, and further supports dissociable neural implementations for these two explorative strategies.
Identifiants
pubmed: 39433838
doi: 10.1038/s41598-024-76025-5
pii: 10.1038/s41598-024-76025-5
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
24796Informations de copyright
© 2024. The Author(s).
Références
Cohen, J. D., McClure, S. M. & Yu, A. J. Should I stay or should I go? How the human brain manages the trade-off between exploitation and exploration. Philos. Trans. R. Soc. B: Biol. Sci. 362, 933–942 (2007).
doi: 10.1098/rstb.2007.2098
Wilson, R. C., Geana, A., White, J. M., Ludvig, E. A. & Cohen, J. D. Humans use directed and random exploration to solve the explore–exploit dilemma. J. Exp. Psychol. Gen. 143, 2074–2081 (2014).
pubmed: 25347535
pmcid: 5635655
doi: 10.1037/a0038199
Auer, P., Cesa-Bianchi, N. & Fischer, P. Finite-time analysis of the multiarmed bandit problem. Mach. Learn. 47, 235–256 (2002).
doi: 10.1023/A:1013689704352
Feng, S. F., Wang, S., Zarnescu, S. & Wilson, R. C. The dynamics of explore–exploit decisions reveal a signal-to-noise mechanism for random exploration. Sci. Rep. 11, 3077 (2021).
pubmed: 33542333
pmcid: 7862437
doi: 10.1038/s41598-021-82530-8
Tomov, M. S., Truong, V. Q., Hundia, R. A. & Gershman, S. J. Dissociable neural correlates of uncertainty underlie different exploration strategies. Nat. Commun. 11, 2371 (2020).
pubmed: 32398675
pmcid: 7217879
doi: 10.1038/s41467-020-15766-z
Averbeck, B. B. Theory of choice in bandit, Information Sampling and foraging tasks. PLoS Comput. Biol. 11, e1004164 (2015).
pubmed: 25815510
pmcid: 4376795
doi: 10.1371/journal.pcbi.1004164
Thompson, W. R., On the likelihood that one unknown probability,, exceeds another in view of the evidence of two samples. Biometrika 25, 285–294 (1933).
doi: 10.1093/biomet/25.3-4.285
Gershman, S. J. Deconstructing the human algorithms for exploration. Cognition 173, 34–42 (2018).
pubmed: 29289795
doi: 10.1016/j.cognition.2017.12.014
Jahn, C. I. et al. Neural responses in macaque prefrontal cortex are linked to strategic exploration. PLoS Biol. 21, e3001985 (2023).
pubmed: 36716348
pmcid: 9910800
doi: 10.1371/journal.pbio.3001985
Badre, D., Doll, B. B., Long, N. M. & Frank, M. J. Rostrolateral prefrontal cortex and individual differences in uncertainty-driven exploration. Neuron 73, 595–607 (2012).
pubmed: 22325209
pmcid: 3285405
doi: 10.1016/j.neuron.2011.12.025
Mansouri, F. A., Koechlin, E., Rosa, M. G. P. & Buckley, M. J. Managing competing goals — a key role for the frontopolar cortex. Nat. Rev. Neurosci. 18, 645–657 (2017).
pubmed: 28951610
doi: 10.1038/nrn.2017.111
Zajkowski, W. K., Kossut, M. & Wilson, R. C. A causal role for right frontopolar cortex in directed, but not random, exploration. Elife 6, e27430 (2017).
Cogliati Dezza, I., Cleeremans, A. & Alexander, W. H. Independent and interacting value systems for reward and information in the human brain. Elife 11, e66358 (2022).
Kobayashi, K. & Kable, J. W. Neural mechanisms of information seeking. Neuron 112, 1741–1756 (2024).
pubmed: 38703774
doi: 10.1016/j.neuron.2024.04.008
Wilson, R. C., Bonawitz, E., Costa, V. D. & Ebitz, R. B. Balancing exploration and exploitation with information and randomization. Curr. Opin. Behav. Sci. 38, 49–56 (2021).
pubmed: 33184605
doi: 10.1016/j.cobeha.2020.10.001
Menon, V. & D’Esposito, M. The role of PFC networks in cognitive control and executive function. Neuropsychopharmacology 47, 90–103 (2022).
pubmed: 34408276
doi: 10.1038/s41386-021-01152-w
Zhen, S., Yaple, Z. A., Eickhoff, S. B. & Yu, R. To learn or to gain: neural signatures of exploration in human decision-making. Brain Struct. Funct. 227, 63–76 (2022).
pubmed: 34596757
doi: 10.1007/s00429-021-02389-3
Huettel, S. A., Song, A. W. & McCarthy, G. Decisions under uncertainty: probabilistic context influences activation of prefrontal and parietal cortices. J. Neurosci. 25, 3304–3311 (2005).
pubmed: 15800185
pmcid: 6724903
doi: 10.1523/JNEUROSCI.5070-04.2005
Hogeveen, J. et al. The neurocomputational bases of explore-exploit decision-making. Neuron 110, 1869-1879.e5 (2022).
pubmed: 35390278
pmcid: 9167768
doi: 10.1016/j.neuron.2022.03.014
Tang, H. & Averbeck, B. B. Shared mechanisms mediate the explore-exploit tradeoff in macaques and humans. Neuron 110, 1751–1753 (2022).
pubmed: 35654023
pmcid: 9589244
doi: 10.1016/j.neuron.2022.05.008
Cho, S. S. et al. Continuous theta burst stimulation of right dorsolateral prefrontal cortex induces changes in impulsivity level. Brain Stimul. 3, 170–176 (2010).
pubmed: 20633446
doi: 10.1016/j.brs.2009.10.002
Cho, S. S. et al. Effect of continuous theta burst stimulation of the right dorsolateral prefrontal cortex on cerebral blood flow changes during decision making. Brain Stimul. 5, 116–123 (2012).
pubmed: 22494829
pmcid: 3707841
doi: 10.1016/j.brs.2012.03.007
Dantas, A. M., Sack, A. T., Bruggen, E., Jiao, P. & Schuhmann, T. The functional relevance of right DLPFC and VMPFC in risk-taking behavior. Cortex 159, 64–74 (2023).
pubmed: 36608421
doi: 10.1016/j.cortex.2022.11.009
Obeso, I., Herrero, M.-T., Ligneul, R., Rothwell, J. C. & Jahanshahi, M. A causal role for the right dorsolateral prefrontal cortex in avoidance of risky choices and making advantageous selections. Neuroscience 458, 166–179 (2021).
pubmed: 33476698
doi: 10.1016/j.neuroscience.2020.12.035
Ngetich, R., Zhou, J., Zhang, J., Jin, Z. & Li, L. Assessing the effects of continuous theta burst stimulation over the dorsolateral prefrontal cortex on human cognition: a systematic review. Front. Integr. Neurosci. 14, 35 (2020).
Knoch, D. et al. Disruption of right Prefrontal cortex by Low-Frequency Repetitive Transcranial Magnetic Stimulation induces risk-taking behavior. J. Neurosci. 26, 6469–6472 (2006).
pubmed: 16775134
pmcid: 6674035
doi: 10.1523/JNEUROSCI.0804-06.2006
Dubois, M. et al. Human complex exploration strategies are enriched by noradrenaline-modulated heuristics. Elife 10, (2021).
Schulz, E. & Gershman, S. J. The algorithmic architecture of exploration in the human brain. Curr. Opin. Neurobiol. 55, 7–14 (2019).
pubmed: 30529148
doi: 10.1016/j.conb.2018.11.003
Chakroun, K., Mathar, D., Wiehler, A., Ganzer, F. & Peters, J. Dopaminergic modulation of the exploration/exploitation trade-off in human decision-making. Elife 9, e51260 (2020).
Wyatt, L. E., Hewan, P. A., Hogeveen, J., Spreng, R. N. & Turner, G. R. Exploration versus exploitation decisions in the human brain: a systematic review of functional neuroimaging and neuropsychological studies. Neuropsychologia 192, 108740 (2024).
pubmed: 38036246
doi: 10.1016/j.neuropsychologia.2023.108740
Sutton, R. S. & Barto, A. G. Reinforcement Learning: An Introduction. (MIT Press, 2018).
Chou, K.-P., Wilson, R. & Smith, R. The influence of anxiety on exploration: a review of computational modeling studies. Preprint at https://doi.org/10.31234/OSF.IO/WTFR6 (2024)
Hogeveen, J. et al. What does the frontopolar cortex contribute to goal-directed cognition and action? J. Neurosci. 42, 8508–8513 (2022).
Dubois, M. & Hauser, T. U. Value-free random exploration is linked to impulsivity. Nat. Commun. 13, 4542 (2022).
pubmed: 35927257
pmcid: 9352791
doi: 10.1038/s41467-022-31918-9
Lin, Y. & Feng, T. Lateralization of self-control over the dorsolateral prefrontal cortex in decision-making: a systematic review and meta-analytic evidence from noninvasive brain stimulation. Cogn. Affect. Behav. Neurosci. 24, 19–41 (2024).
pubmed: 38212486
doi: 10.3758/s13415-023-01148-7
Sadeghiyeh, H. et al. Temporal discounting correlates with directed exploration but not with random exploration. Sci. Rep. 10, 4020 (2020).
pubmed: 32132573
pmcid: 7055215
doi: 10.1038/s41598-020-60576-4
Mohr, P. N. C., Biele, G. & Heekeren, H. R. Neural processing of risk. J. Neurosci. 30, 6613–6619 (2010).
pubmed: 20463224
pmcid: 6632558
doi: 10.1523/JNEUROSCI.0003-10.2010
Spreng, R. N. & Turner, G. R. From exploration to exploitation: a shifting mental mode in late life development. Trends Cogn. Sci. 25, 1058–1071 (2021).
pubmed: 34593321
pmcid: 8844884
doi: 10.1016/j.tics.2021.09.001
Friedman, N. P. & Robbins, T. W. The role of prefrontal cortex in cognitive control and executive function. Neuropsychopharmacology 47, 72–89 (2022).
pubmed: 34408280
doi: 10.1038/s41386-021-01132-0
Laureiro-Martínez, D., Brusoni, S., Canessa, N. & Zollo, M. Understanding the exploration–exploitation dilemma: an < scp > fMRI study of attention control and decision‐making performance. Strateg. Manage. J. 36, 319–338 (2015).
doi: 10.1002/smj.2221
Tobler, P. N., O’Doherty, J. P., Dolan, R. J. & Schultz, W. Reward value coding distinct from risk attitude-related uncertainty coding in human reward systems. J. Neurophysiol. 97, 1621–1632 (2007).
pubmed: 17122317
doi: 10.1152/jn.00745.2006
Romero, M. C., Davare, M., Armendariz, M. & Janssen, P. Neural effects of transcranial magnetic stimulation at the single-cell level. Nat. Commun. 10, 2642 (2019).
pubmed: 31201331
pmcid: 6572776
doi: 10.1038/s41467-019-10638-7
Cathomas, F. et al. Increased random exploration in schizophrenia is associated with inflammation. NPJ Schizophr. 7, 6 (2021).
pubmed: 33536449
pmcid: 7859392
doi: 10.1038/s41537-020-00133-0
Speers, L. J. & Bilkey, D. K. Maladaptive explore/exploit trade-offs in schizophrenia. Trends Neurosci. 46, 341–354 (2023).
pubmed: 36878821
doi: 10.1016/j.tins.2023.02.001
Strauss, G. P. et al. Deficits in positive reinforcement learning and uncertainty-driven exploration are Associated with distinct aspects of negative symptoms in Schizophrenia. Biol. Psychiatry 69, 424–431 (2011).
pubmed: 21168124
doi: 10.1016/j.biopsych.2010.10.015
Blanco, N. J., Otto, A. R., Maddox, W. T., Beevers, C. G. & Love, B. C. The influence of depression symptoms on exploratory decision-making. Cognition 129, 563–568 (2013).
pubmed: 24055832
doi: 10.1016/j.cognition.2013.08.018
Smith, R., Taylor, S., Wilson, R. C., Chuning, A. E., Persich, M. R., Wang, S. & Killgore, W. D. S. Lower levels of directed exploration and reflective thinking are associated with greater anxiety and depression. Front. Psychiatry 12, 782136 (2022).
Fan, H., Gershman, S. J. & Phelps, E. A. Trait somatic anxiety is associated with reduced directed exploration and underestimation of uncertainty. Nat. Hum. Behav. 7, 102–113 (2022).
pubmed: 36192493
doi: 10.1038/s41562-022-01455-y
Aberg, K. C., Toren, I. & Paz, R. A neural and behavioral trade-off between value and uncertainty underlies exploratory decisions in normative anxiety. Mol. Psychiatry 27, 1573–1587 (2022).
pubmed: 34725456
doi: 10.1038/s41380-021-01363-z
Addicott, M. A., Pearson, J. M., Sweitzer, M. M., Barack, D. L. & Platt, M. L. A primer on foraging and the explore/exploit trade-off for psychiatry research. Neuropsychopharmacology 42, 1931–1939 (2017).
pubmed: 28553839
pmcid: 5561336
doi: 10.1038/npp.2017.108
Jung, J., Bungert, A., Bowtell, R. & Jackson, S. R. Vertex stimulation as a control site for transcranial magnetic stimulation: a concurrent TMS/fMRI study. Brain Stimul. 9, 58–64 (2016).
pubmed: 26508284
pmcid: 4720218
doi: 10.1016/j.brs.2015.09.008
Faul, F., Erdfelder, E., Lang, A.-G. & Buchner, A. G*Power 3: a flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behav. Res. Methods 39, 175–191 (2007).
pubmed: 17695343
doi: 10.3758/BF03193146
Mitra, S., Mehta, U. M., Binukumar, B., Venkatasubramanian, G. & Thirthalli, J. Statistical power estimation in non-invasive brain stimulation studies and its clinical implications: an exploratory study of the meta-analyses. Asian J. Psychiatr. 44, 29–34 (2019).
pubmed: 31302440
pmcid: 7610509
doi: 10.1016/j.ajp.2019.07.006
Vékony, T. et al. Continuous theta-burst stimulation over the dorsolateral prefrontal cortex inhibits improvement on a working memory task. Sci. Rep. 8, 14835 (2018).
pubmed: 30287868
pmcid: 6172210
doi: 10.1038/s41598-018-33187-3
Huang, Y.-Z., Edwards, M. J., Rounis, E., Bhatia, K. P. & Rothwell, J. C. Theta burst stimulation of the human motor cortex. Neuron 45, 201–206 (2005).
pubmed: 15664172
doi: 10.1016/j.neuron.2004.12.033
Panidi, K., Vorobiova, A. N., Feurra, M. & Klucharev, V. Dorsolateral prefrontal cortex plays causal role in probability weighting during risky choice. Sci. Rep. 12, 16115 (2022).
pubmed: 36167703
pmcid: 9515118
doi: 10.1038/s41598-022-18529-6
Hanlon, C. A. & McCalley, D. M. Sex/Gender as a factor that influences transcranial magnetic stimulation treatment outcome: three potential biological explanations. Front. Psychiatry 13, (2022).