Dopaminergic reinforcement in the motor system: Implications for Parkinson's disease and deep brain stimulation.
Parkinson's disease
basal ganglia
closed-loop deep brain stimulation
dopamine
neural reinforcement
Journal
The European journal of neuroscience
ISSN: 1460-9568
Titre abrégé: Eur J Neurosci
Pays: France
ID NLM: 8918110
Informations de publication
Date de publication:
04 Jan 2024
04 Jan 2024
Historique:
revised:
16
11
2023
received:
19
09
2023
accepted:
17
11
2023
medline:
5
1
2024
pubmed:
5
1
2024
entrez:
5
1
2024
Statut:
aheadofprint
Résumé
Millions of people suffer from dopamine-related disorders spanning disturbances in movement, cognition and emotion. These changes are often attributed to changes in striatal dopamine function. Thus, understanding how dopamine signalling in the striatum and basal ganglia shapes human behaviour is fundamental to advancing the treatment of affected patients. Dopaminergic neurons innervate large-scale brain networks, and accordingly, many different roles for dopamine signals have been proposed, such as invigoration of movement and tracking of reward contingencies. The canonical circuit architecture of cortico-striatal loops sparks the question, of whether dopamine signals in the basal ganglia serve an overarching computational principle. Such a holistic understanding of dopamine functioning could provide new insights into symptom generation in psychiatry to neurology. Here, we review the perspective that dopamine could bidirectionally control neural population dynamics, increasing or decreasing their strength and likelihood to reoccur in the future, a process previously termed neural reinforcement. We outline how the basal ganglia pathways could drive strengthening and weakening of circuit dynamics and discuss the implication of this hypothesis on the understanding of motor signs of Parkinson's disease (PD), the most frequent dopaminergic disorder. We propose that loss of dopamine in PD may lead to a pathological brain state where repetition of neural activity leads to weakening and instability, possibly explanatory for the fact that movement in PD deteriorates with repetition. Finally, we speculate on how therapeutic interventions such as deep brain stimulation may be able to reinstate reinforcement signals and thereby improve treatment strategies for PD in the future.
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Subventions
Organisme : European Research Council
ID : 101077060
Pays : International
Informations de copyright
© 2023 The Authors. European Journal of Neuroscience published by Federation of European Neuroscience Societies and John Wiley & Sons Ltd.
Références
Albin, R. L., Young, A. B., & Penney, J. B. (1995). The functional anatomy of disorders of the basal ganglia. Trends in Neurosciences, 18, 63-64. https://doi.org/10.1016/0166-2236(95)80020-3
Alexander, G. E., DeLong, M. R., & Strick, P. L. (1986). Parallel organization of functionally segregated circuits linking basal ganglia and cortex. Annual Review of Neuroscience, 9, 357-381. https://doi.org/10.1146/annurev.ne.09.030186.002041
Amaya, K. A., & Smith, K. S. (2018). Neurobiology of habit formation. Current Opinion in Behavioral Sciences Habits and Skills, 20, 145-152.
Athalye, V. R., Carmena, J. M., & Costa, R. M. (2020). Neural reinforcement: Re-entering and refining neural dynamics leading to desirable outcomes. Current Opinion in Neurobiology, 60, 145-154. https://doi.org/10.1016/j.conb.2019.11.023
Barter, J. W., Li, S., Lu, D., Bartholomew, R. A., Rossi, M. A., Shoemaker, C. T., Salas-Meza, D., Gaidis, E., & Yin, H. H. (2015). Beyond reward prediction errors: The role of dopamine in movement kinematics. Frontiers in Integrative Neuroscience, 9, 1-22. https://doi.org/10.3389/fnint.2015.00039
Baunez, C., Nieoullon, A., & Amalric, M. (1995). Dopamine and complex sensorimotor integration: Further studies in a conditioned motor task in the rat. Neuroscience, 65, 375-384. https://doi.org/10.1016/0306-4522(94)00498-T
Beninger, R. J. (1983). The role of dopamine in locomotor activity and learning. Brain Research Reviews, 6, 173-196. https://doi.org/10.1016/0165-0173(83)90038-3
Bergman, H., Wichmann, T., & DeLong, M. R. (1990). Reversal of experimental parkinsonism by lesions of the subthalamic nucleus. Science, 249, 1436-1438. https://doi.org/10.1126/science.2402638
Berridge, K. C. (2007). The debate over dopamine's role in reward: The case for incentive salience. Psychopharmacology, 191, 391-431. https://doi.org/10.1007/s00213-006-0578-x
Bromberg-Martin, E. S., Matsumoto, M., & Hikosaka, O. (2010). Dopamine in motivational control: Rewarding, aversive, and alerting. Neuron, 68, 815-834. https://doi.org/10.1016/j.neuron.2010.11.022
Capogrosso, M., Milekovic, T., Borton, D., Wagner, F., Moraud, E. M., Mignardot, J.-B., Buse, N., Gandar, J., Barraud, Q., Xing, D., Rey, E., Duis, S., Jianzhong, Y., Ko, W. K. D., Li, Q., Detemple, P., Denison, T., Micera, S., Bezard, E., … Courtine, G. (2016). A brain-spine interface alleviating gait deficits after spinal cord injury in primates. Nature, 539, 284-288. https://doi.org/10.1038/nature20118
Cheung, T. H. C., Ding, Y., Zhuang, X., & Kang, U. J. (2023). Learning critically drives parkinsonian motor deficits through imbalanced striatal pathway recruitment. Proceedings of the National Academy of Sciences, 120, e2213093120. https://doi.org/10.1073/pnas.2213093120
Chu, C., Liu, S., He, N., Zeng, Z., Wang, J., Zhang, Z., Zeljic, K., van der Stelt, O., Sun, B., Yan, F., Liu, C., Li, D., & Zhang, C. (2023). Subthalamic stimulation modulates motor network in Parkinson's disease: Recover, relieve and remodel. Brain, 146, 2780-2791. https://doi.org/10.1093/brain/awad004
Coddington, L. T., & Dudman, J. T. (2019). Learning from action: Reconsidering movement signaling in midbrain dopamine neuron activity. Neuron, 104, 63-77. https://doi.org/10.1016/j.neuron.2019.08.036
Coddington, L. T., Lindo, S. E., & Dudman, J. T. (2023). Mesolimbic dopamine adapts the rate of learning from action. Nature, 614, 294-302.
Cui, G., Jun, S. B., Jin, X., Pham, M. D., Vogel, S. S., Lovinger, D. M., & Costa, R. M. (2013). Concurrent activation of striatal direct and indirect pathways during action initiation. Nature, 494, 238-242. https://doi.org/10.1038/nature11846
Daniel, R., & Pollmann, S. (2014). A universal role of the ventral striatum in reward-based learning: Evidence from human studies. Neurobiology of Learning and Memory, 0, 90-100. https://doi.org/10.1016/j.nlm.2014.05.002
De Almeida Marcelino, A. L., Horn, A., Krause, P., Kühn, A. A., & Neumann, W. J. (2019). Subthalamic neuromodulation improves short-term motor learning in Parkinson's disease. Brain, 142, 2198-2206. https://doi.org/10.1093/brain/awz152
Dodson, P. D., Dreyer, J. K., Jennings, K. A., Syed, E. C. J., Wade-Martins, R., Cragg, S. J., Bolam, J. P., & Magill, P. J. (2016). Representation of spontaneous movement by dopaminergic neurons is cell-type selective and disrupted in parkinsonism. Proceedings of the National Academy of Sciences of the United States of America, 113, E2180-E2188. https://doi.org/10.1073/pnas.1515941113
Dorsey, E. R., Sherer, T., Okun, M. S., & Bloem, B. R. (2018). The emerging evidence of the Parkinson pandemic. Journal of Parkinson's Disease, 8, 3-8. https://doi.org/10.3233/JPD-181474
Feingold, J., Gibson, D. J., DePasquale, B., & Graybiel, A. M. (2015). Bursts of beta oscillation differentiate postperformance activity in the striatum and motor cortex of monkeys performing movement tasks. Proceedings of the National Academy of Sciences, 112, 13687-13692. https://doi.org/10.1073/pnas.1517629112
Fishman, R. H. B., Feigenbaum, J. J., Yanai, J., & Klawans, H. L. (1983). The relative importance of dopamine and norepinephrine in mediating locomotor activity. Progress in Neurobiology, 20, 55-88. https://doi.org/10.1016/0301-0082(83)90010-2
Foffani, G., & Obeso, J. A. (2018). A cortical pathogenic theory of Parkinson's disease. Neuron, 99, 1116-1128. https://doi.org/10.1016/j.neuron.2018.07.028
Glennon, E., Svirsky, M. A., & Froemke, R. C. (2020). Auditory cortical plasticity in cochlear implant users. Current Opinion in Neurobiology, 60, 108-114. https://doi.org/10.1016/j.conb.2019.11.003
Grace, A. A., Floresco, S. B., Goto, Y., & Lodge, D. J. (2007). Regulation of firing of dopaminergic neurons and control of goal-directed behaviors. Trends in Neurosciences, 30, 220-227. https://doi.org/10.1016/j.tins.2007.03.003
Greenstreet, F., Vergara, H.M., Pati, S., Schwarz, L., Wisdom, M., Marbach, F., Johansson, Y., Rollik, L., Moskovitz, T., Clopath, C., & Stephenson-Jones, M. (2022) Action prediction error: A value-free dopaminergic teaching signal that drives stable learning. bioRxiv, 2022.09.12.507572.
Hallett, M. (1990). Clinical neurophysiology of akinesia. Revue Neurologique (Paris), 146, 585-590.
He, S., Baig, F., Merla, A., Torrecillos, F., Perera, A., Wiest, C., Debarros, J., Benjaber, M., Hart, M. G., Ricciardi, L., Morgante, F., Hasegawa, H., Samuel, M., Edwards, M., Denison, T., Pogosyan, A., Ashkan, K., Pereira, E., & Tan, H. (2023). Beta-triggered adaptive deep brain stimulation during reaching movement in Parkinson's disease. Brain, 146(12), 5015-5030. https://doi.org/10.1093/brain/awad233
Kehnemouyi, Y. M., Petrucci, M. N., Wilkins, K. B., Melbourne, J. A., & Bronte-Stewart, H. M. (2023). The sequence effect worsens over time in Parkinson's disease and responds to open and closed-loop subthalamic nucleus deep brain stimulation. Journal of Parkinson's Disease, 13, 537-548. https://doi.org/10.3233/JPD-223368
Köhler, R.M., Binns, T.S., Merk, T., Zhu, G., Yin, Z., Zhao, B., Chikermane, M., Vanhoecke, J., Busch, J.L., Habets, J., Faust, K., Schneider, G.-H., Haufe, S., Zhang, J., Kühn, A.A., Haynes, J.-D., & Neumann, W.-J. (2023) Dopamine and neuromodulation shorten the latency from motor intention to execution in Parkinson's disease.
Kravitz, A. V., Tye, L. D., & Kreitzer, A. C. (2012). Distinct roles for direct and indirect pathway striatal neurons in reinforcement. Nature Neuroscience, 15, 816-818. https://doi.org/10.1038/nn.3100
Kwon, D. K., Kwatra, M., Wang, J., & Ko, H. S. (2022). Levodopa-induced dyskinesia in Parkinson's disease: Pathogenesis and emerging treatment strategies. Cell, 11, 3736. https://doi.org/10.3390/cells11233736
Ling, H., Massey, L. A., Lees, A. J., Brown, P., & Day, B. L. (2012). Hypokinesia without decrement distinguishes progressive supranuclear palsy from Parkinson's disease. Brain, 135, 1141-1153. https://doi.org/10.1093/brain/aws038
Lofredi, R., Okudzhava, L., Irmen, F., Brücke, C., Huebl, J., Krauss, J. K., Schneider, G.-H., Faust, K., Neumann, W.-J., & Kühn, A. A. (2023). Subthalamic beta bursts correlate with dopamine-dependent motor symptoms in 106 Parkinson's patients. NPJ Parkinson's Disease, 9, 1-9.
Markowitz, J. E., Gillis, W. F., Jay, M., Wood, J., Harris, R. W., Cieszkowski, R., Scott, R., Brann, D., Koveal, D., Kula, T., Weinreb, C., Osman, M. A. M., Pinto, S. R., Uchida, N., Linderman, S. W., Sabatini, B. L., & Datta, S. R. (2023). Spontaneous behaviour is structured by reinforcement without explicit reward. Nature, 2023, 1-10. https://doi.org/10.1038/s41586-022-05611-2
Marsden, C. D., & Obeso, J. A. (1994). The functions of the basal ganglia and the paradox of stereotaxic surgery in Parkinson's disease. Brain: a Journal of Neurology, 117(Pt 4), 877-897. https://doi.org/10.1093/brain/117.4.877
Marshall, J. F., Levitan, D., & Stricker, E. M. (1976). Activation-induced restoration of sensorimotor functions in rats with dopamine-depleting brain lesions. Journal of Comparative and Physiological Psychology, 90, 536-546. https://doi.org/10.1037/h0077230
Merk, T., Köhler, R., Peterson, V., Lyra, L., Vanhoecke, J., Chikermane, M., Binns, T., Li, N., Walton, A., Bush, A., Sisterson, N., Busch, J., Lofredi, R., Habets, J., Huebl, J., Zhu, G., Yin, Z., Zhao, B., Merkl, A., … Neumann, W.-J. (2023). Invasive neurophysiology and whole brain connectomics for neural decoding in patients with brain implants. Research Square, 1-30. https://doi.org/10.21203/rs.3.rs-3212709/v1
Merk, T., Peterson, V., Lipski, W. J., Blankertz, B., Turner, R. S., Li, N., Horn, A., Richardson, R. M., & Neumann, W.-J. (2022). Electrocorticography is superior to subthalamic local field potentials for movement decoding in Parkinson's disease. eLife, 11, e75126. https://doi.org/10.7554/eLife.75126
Miller, W. C., & DeLong, M. R. (1987). Altered tonic activity of neurons in the globus pallidus and subthalamic nucleus in the primate MPTP model of Parkinsonism. In M. B. Carpenter & A. Jayaraman (Eds.), The basal ganglia II, advances in behavioral biology (pp. 415-427). Springer US. https://doi.org/10.1007/978-1-4684-5347-8_29
Mills, J. O., Jalil, A., & Stanga, P. E. (2017). Electronic retinal implants and artificial vision: Journey and present. Eye, 31, 1383-1398. https://doi.org/10.1038/eye.2017.65
Milosevic, L., Kalia, S. K., Hodaie, M., Lozano, A. M., Fasano, A., Popovic, M. R., & Hutchison, W. D. (2018). Neuronal inhibition and synaptic plasticity of basal ganglia neurons in Parkinson's disease. Brain: a Journal of Neurology, 141, 177-190. https://doi.org/10.1093/brain/awx296
Neumann, W.-J., Gilron, R., Little, S., & Tinkhauser, G. (2023). Adaptive deep brain stimulation: From experimental evidence toward practical implementation. Movement Disorders, 38, 937-948. https://doi.org/10.1002/mds.29415
Neumann, W.-J., Horn, A., & Kühn, A. A. (2023). Insights and opportunities for deep brain stimulation as a brain circuit intervention. Trends in Neurosciences, 46, 472-487. https://doi.org/10.1016/j.tins.2023.03.009
Neumann, W.-J., Steiner, L. A., & Milosevic, L. (2023). Neurophysiological mechanisms of deep brain stimulation across spatiotemporal resolutions. Brain, 146(11), 4456-4468. https://doi.org/10.1093/brain/awad239
Nieoullon, A. (2002). Dopamine and the regulation of cognition and attention. Progress in Neurobiology, 67, 53-83. https://doi.org/10.1016/S0301-0082(02)00011-4
Oishi, Y., & Lazarus, M. (2017). The control of sleep and wakefulness by mesolimbic dopamine systems. Neuroscience Reserch Cutting-Edge Approaches to Unwrapping the Mysteries of Sleep, 118, 66-73.
Phillips, A. G., Vacca, G., & Ahn, S. (2008). A top-down perspective on dopamine, motivation and memory. Pharmacology Biochemistry and Behavior, Microdialysis: Recent Developments, 90, 236-249. https://doi.org/10.1016/j.pbb.2007.10.014
Poewe, W., Seppi, K., Tanner, C. M., Halliday, G. M., Brundin, P., Volkmann, J., Schrag, A.-E., & Lang, A. E. (2017). Parkinson disease. Nature Reviews Disease Primers, 3, 1-21.
Potashkin, J. A., Blume, S. R., & Runkle, N. K. (2011). Limitations of animal models of Parkinson's disease. Parkinsons Disease, 2011, 2011-2017. https://doi.org/10.4061/2011/658083
Sabandal, J. M., Berry, J. A., & Davis, R. L. (2021). Dopamine-based mechanism for transient forgetting. Nature, 591, 426-430. https://doi.org/10.1038/s41586-020-03154-y
Salgado-Pineda, P., Delaveau, P., Blin, O., & Nieoullon, A. (2005). Dopaminergic contribution to the regulation of emotional perception. Clinical Neuropharmacology, 28, 228-237. https://doi.org/10.1097/01.wnf.0000185824.57690.f0
Schneider, S., Lee, J. H., & Mathis, M. W. (2023). Learnable latent embeddings for joint behavioural and neural analysis. Nature, 617, 360-368. https://doi.org/10.1038/s41586-023-06031-6
Schuepbach, W. M. M., Rau, J., Knudsen, K., Volkmann, J., Krack, P., Timmermann, L., Hälbig, T. D., Hesekamp, H., Navarro, S. M., Meier, N., Falk, D., Mehdorn, M., Paschen, S., Maarouf, M., Barbe, M. T., Fink, G. R., Kupsch, A., Gruber, D., Schneider, G.-H., … Deuschl, G. (2013). Neurostimulation for Parkinson's disease with early motor complications. New England Journal of Medicine, 368, 610-622. https://doi.org/10.1056/NEJMoa1205158
Schultz, W. (1994). Behavior-related activity of primate dopamine neurons. Revue Neurologique (Paris), 150, 634-639.
Schultz, W., Dayan, P., & Montague, P. R. (1997). A neural substrate of prediction and reward. Science, 275, 1593-1599. https://doi.org/10.1126/science.275.5306.1593
Schwerdt, H. N., Amemori, K., Gibson, D. J., Stanwicks, L. L., Yoshida, T., Bichot, N. P., Amemori, S., Desimone, R., Langer, R., Cima, M. J., & Graybiel, A. M. (2020). Dopamine and beta-band oscillations differentially link to striatal value and motor control. Science Advances, 6, eabb9226. https://doi.org/10.1126/sciadv.abb9226
Shafiei, G., Zeighami, Y., Clark, C. A., Coull, J. T., Nagano-Saito, A., Leyton, M., Dagher, A., & Mišić, B. (2019). Dopamine signaling modulates the stability and integration of intrinsic brain networks. Cerebral Cortex N. Y. NY, 29, 397-409. https://doi.org/10.1093/cercor/bhy264
Shen, W., Flajolet, M., Greengard, P., & Surmeier, D. J. (2008). Dichotomous dopaminergic control of striatal synaptic plasticity. Science, 321, 848-851. https://doi.org/10.1126/science.1160575
Shohamy, D., & Adcock, R. A. (2010). Dopamine and adaptive memory. Trends in Cognitive Sciences, 14, 464-472. https://doi.org/10.1016/j.tics.2010.08.002
Smith, Y., & Kieval, J. Z. (2000). Anatomy of the dopamine system in the basal ganglia. Trends in Neurosciences, 23, S28-S33. https://doi.org/10.1016/S1471-1931(00)00023-9
Stephenson-Jones, M., Samuelsson, E., Ericsson, J., Robertson, B., & Grillner, S. (2011). Evolutionary conservation of the basal ganglia as a common vertebrate mechanism for action selection. Current Biology, 21, 1081-1091. https://doi.org/10.1016/j.cub.2011.05.001
Tambasco, N., Romoli, M., & Calabresi, P. (2018). Levodopa in Parkinson's disease: Current status and future developments. Current Neuropharmacology, 16, 1239-1252. https://doi.org/10.2174/1570159X15666170510143821
Tecuapetla, F., Jin, X., Lima, S. Q., & Costa, R. M. (2016). Complementary contributions of striatal projection pathways to action initiation and execution. Cell, 166, 703-715. https://doi.org/10.1016/j.cell.2016.06.032
Turner, R. S., & Desmurget, M. (2010). Basal ganglia contributions to motor control: A vigorous tutor. Current Opinion in Neurobiology, 20, 704-716. https://doi.org/10.1016/j.conb.2010.08.022
van der Hoorn, A., Burger, H., Leenders, K. L., & de Jong, B. M. (2012). Handedness correlates with the dominant Parkinson side: A systematic review and meta-analysis. Movement Disorders, 27, 206-210. https://doi.org/10.1002/mds.24007
Wang, D. V., & Tsien, J. Z. (2011). Conjunctive processing of locomotor signals by the ventral tegmental area neuronal population. PLoS ONE, 6, e16528. https://doi.org/10.1371/journal.pone.0016528
Wichmann, T., Bergman, H., & DeLong, M. R. (2018). Basal ganglia, movement disorders and deep brain stimulation: Advances made through non-human primate research. Journal of Neural Transmission, 125, 419-430. https://doi.org/10.1007/s00702-017-1736-5
Wood, A. N. (2021). New roles for dopamine in motor skill acquisition: Lessons from primates, rodents, and songbirds. Journal of Neurophysiology, 125, 2361-2374. https://doi.org/10.1152/jn.00648.2020
Yttri, E. A., & Dudman, J. T. (2016). Opponent and bidirectional control of movement velocity in the basal ganglia. Nature, 533, 402-406. https://doi.org/10.1038/nature17639