Human-centred physical neuromorphics with visual brain-computer interfaces.
Journal
Nature communications
ISSN: 2041-1723
Titre abrégé: Nat Commun
Pays: England
ID NLM: 101528555
Informations de publication
Date de publication:
29 Jul 2024
29 Jul 2024
Historique:
received:
14
11
2023
accepted:
19
07
2024
medline:
31
7
2024
pubmed:
31
7
2024
entrez:
30
7
2024
Statut:
epublish
Résumé
Steady-state visual evoked potentials (SSVEPs) are widely used for brain-computer interfaces (BCIs) as they provide a stable and efficient means to connect the computer to the brain with a simple flickering light. Previous studies focused on low-density frequency division multiplexing techniques, i.e. typically employing one or two light-modulation frequencies during a single flickering light stimulation. Here we show that it is possible to encode information in SSVEPs excited by high-density frequency division multiplexing, involving hundreds of frequencies. We then demonstrate the ability to transmit entire images from the computer to the brain/EEG read-out in relatively short times. High-density frequency multiplexing also allows to implement a photonic neural network utilizing SSVEPs, that is applied to simple classification tasks and exhibits promising scalability properties by connecting multiple brains in series. Our findings open up new possibilities for the field of neural interfaces, holding potential for various applications, including assistive technologies and cognitive enhancements, to further improve human-machine interactions.
Identifiants
pubmed: 39080312
doi: 10.1038/s41467-024-50775-2
pii: 10.1038/s41467-024-50775-2
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
6393Subventions
Organisme : RCUK | Engineering and Physical Sciences Research Council (EPSRC)
ID : EP/T021020/1
Informations de copyright
© 2024. The Author(s).
Références
Cheng, M., Gao, X., Gao, S. & Xu, D. Design and implementation of a brain–computer interface with high transfer rates. IEEE Trans. Biomed. Eng. 49, 1181 (2002).
doi: 10.1109/TBME.2002.803536
pubmed: 12374343
Bi, L., Fan, X. A. & Liu, Y. EEG-based brain-controlled mobile robots: a survey. IEEE Trans. Hum.–Mach. Syst. 43, 161 (2013).
doi: 10.1109/TSMCC.2012.2219046
Nagel, S. & Spüler, M. World’s fastest brain-computer interface: combining eeg2code with deeplearning. PLoS ONE 14, e0221909 (2019).
doi: 10.1371/journal.pone.0221909
pubmed: 31490999
pmcid: 6730910
Chen, X. et al. High-speed spelling with a noninvasive brain-computer interface. PNAS 112, E6058–E6067 (2015).
doi: 10.1073/pnas.1508080112
pubmed: 26483479
pmcid: 4640776
Vialatte, F. B., Maurice, M., Dauwels, J. & Cichocki, A. Steady-state visually evoked potentials: Focus on essential paradigms and future perspectives. Prog. Neurobiol. 90, 418 (2010).
doi: 10.1016/j.pneurobio.2009.11.005
pubmed: 19963032
Norcia, A. M., Appelbaum, L. G., Ales, J. M., Cottereau, B. R. & Rossion, B. The steady-state visual evoked potential in vision research: a review. J. Vis. 15, 4 (2015).
doi: 10.1167/15.6.4
pubmed: 26024451
pmcid: 4581566
Zemon, V. & Ratliff, F. Intermodulation components of the visual evoked potential: responses to lateral and superimposed stimuli. Biol. Cybern. 50, 401 (1984).
doi: 10.1007/BF00335197
pubmed: 6487677
Regan, M. & Regan, D. A frequency domain technique for characterizing nonlinearities in biological systems. J. Theor. Biol. 133, 293 (1988).
doi: 10.1016/S0022-5193(88)80323-0
Çetin, V., Ozekes, S. & Varol, H. S. Harmonic analysis of steady-state visual evoked potentials in brain computer interfaces. Biomed. Signal Process. Control 60, 101999 (2020).
doi: 10.1016/j.bspc.2020.101999
Heinrichs-Graham, E. & Wilson, T. Presence of strong harmonics during visual entrainment: a magnetoencephalography study. J. Theor. Biol. 91, 59 (2012).
Gordon, N., Hohwy, J., Davidson, M. J., van Boxtel, J. J. & Tsuchiya, N. From intermodulation components to visual perception and cognition—a review. NeuroImage 199, 480–494 (2019).
doi: 10.1016/j.neuroimage.2019.06.008
pubmed: 31173903
Labecki, M. et al. Nonlinear origin of ssvep spectra—a combined experimental and modeling study. Front. Comput. Neurosci. 10, 129 (2016).
doi: 10.3389/fncom.2016.00129
pubmed: 28082888
pmcid: 5187367
Luff, C. E. et al. The neuron mixer and its impact on human brain dynamics. Cell Rep. 43, 114274 (2024).
Mu, J., Grayden, D. B., Tan, Y. & Oetomo, D. Frequency set selection for multi-frequency steady-state visual evoked potential-based brain-computer interfaces. Front. Neurosci. 16, 1057010 (2022).
doi: 10.3389/fnins.2022.1057010
pubmed: 36620442
pmcid: 9811191
Shyu, K.-K., Lee, P.-L., Liu, Y.-J. & Sie, J.-J. Dual-frequency steady-state visual evoked potential for brain computer interface. Neurosci. Lett. 483, 28–31 (2010).
doi: 10.1016/j.neulet.2010.07.043
pubmed: 20655362
Mukesh, T. S., Jaganathan, V. & Reddy, M. R. A novel multiple frequency stimulation method for steady state vep based brain computer interfaces. Physiol. Meas. 27, 61 (2005).
doi: 10.1088/0967-3334/27/1/006
Koch, K. et al. How much the eye tells the brain. Curr. Biol. 16, 1428–1434 (2006).
doi: 10.1016/j.cub.2006.05.056
pubmed: 16860742
pmcid: 1564115
Boccolini, A., Fedrizzi, A. & Faccio, D. Ghost imaging with the human eye. Opt. Express 27, 9258 (2019).
doi: 10.1364/OE.27.009258
pubmed: 31052733
Wang, G. & Faccio, D. Computational ghost imaging with the human brain. Intell. Comp. 2, 0014 (2023).
doi: 10.34133/icomputing.0014
LeCun, Y., Cortes, C. & Burges, C. J. C. The MNIST Database of Handwritten Digits http://yann.lecun.com/exdb/mnist/ (1998).
Shen, Y. et al. Deep learning with coherent nanophotonic circuits. Nat. Photon 11, 441–446 (2017).
doi: 10.1038/nphoton.2017.93
der Sande, G. V., Brunner, D. & Soriano, M. C. Advances in photonic reservoir computing. Nanophotonics 6, 561–576 (2017).
doi: 10.1515/nanoph-2016-0132
Bueno, J. et al. Reinforcement learning in a large-scale photonic recurrent neural network. Optica 5, 756–760 (2018).
doi: 10.1364/OPTICA.5.000756
Tanaka, G. et al. Recent advances in physical reservoir computing: a review. Neural Netw. 115, 100 (2019).
doi: 10.1016/j.neunet.2019.03.005
pubmed: 30981085
Rafayelyan, M., Dong, J., Tan, Y., Krzakala, F. & Gigan, S. Large-scale optical reservoir computing for spatiotemporal chaotic systems prediction. Phys. Rev. X 10, 041037 (2020).
Marcucci, G., Pierangeli, D. & Conti, C. Theory of neuromorphic computing by waves: machine learning by rogue waves, dispersive shocks, and solitons. Phys. Rev. Lett. 125, 093901 (2020).
doi: 10.1103/PhysRevLett.125.093901
pubmed: 32915624
Pierangeli, D., Marcucci, G. & Conti, C. Photonic extreme learning machine by free-space optical propagation. Photon. Res. 9, 1446 (2021).
doi: 10.1364/PRJ.423531
Lupo, A., Butschek, L. & Massar, S. Photonic extreme learning machine based on frequency multiplexing. Opt. Express 29, 28257–28276 (2021).
doi: 10.1364/OE.433535
pubmed: 34614961
Butschek, L. et al. Photonic reservoir computer based on frequency multiplexing. Opt. Lett. 47, 782–785 (2022).
doi: 10.1364/OL.451087
pubmed: 35167524
Oguz, I. et al. Programming nonlinear propagation for efficient optical learning machines. Adv. Photonics 6, 016002 (2024).
Wright, L. G. et al. Deep physical neural networks trained with backpropagation. Nature 601, 649 (2022).
doi: 10.1038/s41586-021-04223-6
Xia, F. et al. Hardware-efficient, large-scale reconfigurable optical neural network (ONN) with backpropagation. In AI and Optical Data Sciences IV (eds Jalali, B. & Kitayama, K.) Vol. 12438, 124380Z (SPIE, 2023).
Lupo, A., Picco, E., Zajnulina, M. & Massar, S. Fully analog photonic deep reservoir computer based on frequency multiplexing. Optica 10, 1478 (2023).
Alpaydin, E. & Kaynak, C. Optical Recognition of Handwritten Digits. UCI Machine Learning Repository https://doi.org/10.24432/C50P49 (1998).
Pierro, A., Heiney, K., Shrivastava, S., Marcucci, G. & Nichele, S. Optimization of a hydrodynamic computational reservoir through evolution. In Proc. GECCO ’23 (eds Silva, S. & Paquete, L.) (Association for Computing Machinery, New York, NY, USA, 2023).
Fisher, R. Iris. UCI Machine Learning Repository https://doi.org/10.24432/C56C76 (1988).
Gallicchio, C., Micheli, A. & Pedrelli, L. Deep reservoir computing: a critical experimental analysis. Neurocomputing 268, 87–99 (2017).
doi: 10.1016/j.neucom.2016.12.089
Nichele, S. & Molund, A. Deep reservoir computing using cellular automata. Preprint at https://doi.org/10.48550/arXiv.1703.02806 (2017).
Goldmann, M., Köster, F., Lüdge, K. & Yanchuk, S. Deep time-delay reservoir computing: dynamics and memory capacity. Chaos 30, 093124 (2020).
doi: 10.1063/5.0017974
pubmed: 33003948
Marcucci, G., Caramazza, P. & Shrivastava, S. A new paradigm of reservoir computing exploiting hydrodynamics. Phys. Fluids 35, 071703 (2023).
doi: 10.1063/5.0157919
Boyd, R. W. Nonlinear Optics 4 edn (Academic Press, Elsevier, 2020).