On the role of generative artificial intelligence in the development of brain-computer interfaces.
Brain-computer interface
Diffusion model
Generative adversarial networks
Generative artificial intelligence
Transformers
Variational autoencoders
Journal
BMC biomedical engineering
ISSN: 2524-4426
Titre abrégé: BMC Biomed Eng
Pays: England
ID NLM: 101756092
Informations de publication
Date de publication:
02 May 2024
02 May 2024
Historique:
received:
04
11
2023
accepted:
24
04
2024
medline:
3
5
2024
pubmed:
3
5
2024
entrez:
2
5
2024
Statut:
epublish
Résumé
Since their inception more than 50 years ago, Brain-Computer Interfaces (BCIs) have held promise to compensate for functions lost by people with disabilities through allowing direct communication between the brain and external devices. While research throughout the past decades has demonstrated the feasibility of BCI to act as a successful assistive technology, the widespread use of BCI outside the lab is still beyond reach. This can be attributed to a number of challenges that need to be addressed for BCI to be of practical use including limited data availability, limited temporal and spatial resolutions of brain signals recorded non-invasively and inter-subject variability. In addition, for a very long time, BCI development has been mainly confined to specific simple brain patterns, while developing other BCI applications relying on complex brain patterns has been proven infeasible. Generative Artificial Intelligence (GAI) has recently emerged as an artificial intelligence domain in which trained models can be used to generate new data with properties resembling that of available data. Given the enhancements observed in other domains that possess similar challenges to BCI development, GAI has been recently employed in a multitude of BCI development applications to generate synthetic brain activity; thereby, augmenting the recorded brain activity. Here, a brief review of the recent adoption of GAI techniques to overcome the aforementioned BCI challenges is provided demonstrating the enhancements achieved using GAI techniques in augmenting limited EEG data, enhancing the spatiotemporal resolution of recorded EEG data, enhancing cross-subject performance of BCI systems and implementing end-to-end BCI applications. GAI could represent the means by which BCI would be transformed into a prevalent assistive technology, thereby improving the quality of life of people with disabilities, and helping in adopting BCI as an emerging human-computer interaction technology for general use.
Identifiants
pubmed: 38698495
doi: 10.1186/s42490-024-00080-2
pii: 10.1186/s42490-024-00080-2
doi:
Types de publication
Journal Article
Review
Langues
eng
Pagination
4Informations de copyright
© 2024. The Author(s).
Références
Wolpaw JR, Birbaumer N, McFarland DJ, Pfurtscheller G, Vaughan TM. Brain–computer interfaces for communication and control. Clin Neurophys. 2002;113:767–91.
doi: 10.1016/S1388-2457(02)00057-3
Saha S, Mamun KA, Ahmed K, Mostafa R, Naik GR, Darvishi S, Khandoker AH, Baumert M. Progress in brain computer interface: challenges and opportunities. Front Syst Neurosci. 2021;15:578875.
doi: 10.3389/fnsys.2021.578875
MILLÁN DELR, Ferrez J, Galán PW, Lew F, Chavarriaga E. Non-invasive brain-machine interaction. Int J Pattern Recognit Artif Intell. 2008;22:959–72.
doi: 10.1142/S0218001408006600
Krusienski DJ, Sellers EW, Cabestaing F, Bayoudh S, McFarland DJ, Vaughan TM, Wolpaw JR. A comparison of classification techniques for the P300 speller. J Neural Eng. 2006;3:299.
doi: 10.1088/1741-2560/3/4/007
Krusienski DJ, Sellers EW, McFarland DJ, Vaughan TM, Wolpaw JR. Toward enhanced P300 speller performance. J Neurosci Methods. 2008;167:15–21.
doi: 10.1016/j.jneumeth.2007.07.017
Elsawy AS, Eldawlatly S, Taher M, Aly GM. MindEdit: a P300-based text editor for mobile devices. Comput Biol Med 2017, 80:97–106.
Perdikis S, Leeb R, Williamson J, Ramsay A, Tavella M, Desideri L, Hoogerwerf E-J, Al-Khodairy A, Murray-Smith R., d R Millán J: clinical evaluation of BrainTree, a motor imagery hybrid BCI speller. J Neural Eng. 2014;11:036003.
doi: 10.1088/1741-2560/11/3/036003
Tanaka K, Matsunaga K, Wang HO. Electroencephalogram-Based Control of an Electric Wheelchair. IEEE Trans Robot. 2005;21:762–6.
doi: 10.1109/TRO.2004.842350
Tonin L, Perdikis S, Kuzu TD, Pardo J, Orset B, Lee K, Aach M, Schildhauer TA, Martínez-Olivera R. Millán JdR: learning to control a BMI-driven wheelchair for people with severe tetraplegia. Iscience 2022, 25.
Tariq M, Trivailo PM, Simic M. EEG-based BCI control schemes for lower-limb assistive-robots. Front Hum Neurosci. 2018;12:312.
doi: 10.3389/fnhum.2018.00312
Vidaurre C, Klauer C, Schauer T, Ramos-Murguialday A, Müller K-R. EEG-based BCI for the linear control of an upper-limb neuroprosthesis. Med Eng Phys. 2016;38:1195–204.
doi: 10.1016/j.medengphy.2016.06.010
Baniqued PDE, Stanyer EC, Awais M, Alazmani A, Jackson AE, Mon-Williams MA, Mushtaq F, Holt RJ. Brain–computer interface robotics for hand rehabilitation after stroke: a systematic review. J Neuroeng Rehabil. 2021;18:1–25.
doi: 10.1186/s12984-021-00820-8
Chamola V, Vineet A, Nayyar A, Hossain E. Brain-computer interface-based humanoid control: a review. Sensors. 2020;20:3620.
doi: 10.3390/s20133620
Tonin L, Millán JR. Noninvasive brain–machine interfaces for robotic devices. Annual Rev Control Rob Auton Syst. 2021;4:191–214.
doi: 10.1146/annurev-control-012720-093904
Nijholt A, Bos DO, Reuderink B. Turning shortcomings into challenges: brain-computer interfaces for games. Entertain Comput. 2009;1:85–94.
doi: 10.1016/j.entcom.2009.09.007
Kohli V, Tripathi U, Chamola V, Rout BK, Kanhere SS. A review on virtual reality and augmented reality use-cases of Brain Computer Interface based applications for smart cities. Microprocess Microsyst. 2022;88:104392.
doi: 10.1016/j.micpro.2021.104392
Chavarriaga R, Ušćumlić M, Zhang H, Khaliliardali Z, Aydarkhanov R, Saeedi S, Gheorghe L, Millán JR. Decoding neural correlates of cognitive states to enhance driving experience. IEEE Trans Emerg Top Comput Intell. 2018;2:288–97.
doi: 10.1109/TETCI.2018.2848289
Mustafa Y, Elmahallawy M, Luo T, Eldawlatly S: A brain-computer interface augmented reality framework with auto-adaptive ssvep recognition. In. 2023 IEEE International Conference on Metrology for eXtended Reality, Artificial Intelligence and Neural Engineering (MetroXRAINE). IEEE; 2023: 799–804.
Simon C, Bolton DA, Kennedy NC, Soekadar SR, Ruddy KL. Challenges and opportunities for the future of brain-computer interface in neurorehabilitation. Front NeuroSci. 2021;15:699428.
doi: 10.3389/fnins.2021.699428
Sabio J, Williams NS, McArthur GM, Badcock NA. A scoping review on the use of consumer-grade EEG devices for research. PLoS ONE. 2024;19:e0291186.
doi: 10.1371/journal.pone.0291186
Saha S, Baumert M. Intra-and inter-subject variability in EEG-based sensorimotor brain computer interface: a review. Front Comput Neurosci. 2020;13:87.
doi: 10.3389/fncom.2019.00087
Yadav D, Yadav S, Veer K. A comprehensive assessment of Brain Computer interfaces: recent trends and challenges. J Neurosci Methods. 2020;346:108918.
doi: 10.1016/j.jneumeth.2020.108918
Gozalo-Brizuela R, Garrido-Merchan EC. ChatGPT is not all you need. A state of the art review of large generative AI models. arXiv Preprint arXiv:230104655 2023.
Cao Y, Li S, Liu Y, Yan Z, Dai Y, Yu PS, Sun L. A comprehensive survey of Ai-generated content (aigc): a history of generative ai from gan to chatgpt. arXiv Preprint arXiv:230304226 2023.
Habashi AG, Azab AM, Eldawlatly S, Aly GM. Generative adversarial networks in EEG analysis: an overview. J Neuroeng Rehabil. 2023;20:40.
doi: 10.1186/s12984-023-01169-w
Kingma DP, Welling M. Auto-encoding variational bayes. arXiv Preprint arXiv:13126114 2013.
Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y. Generative adversarial networks. Commun ACM. 2020;63:139–44.
doi: 10.1145/3422622
Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I. Attention is all you need. Adv Neural Inf Process Syst 2017, 30.
Yang L, Zhang Z, Song Y, Hong S, Xu R, Zhao Y, Zhang W, Cui B, Yang M-H. Diffusion models: a comprehensive survey of methods and applications. ACM Computing Surveys; 2022.
Picton TW. The P300 wave of the human event-related potential. J Clin Neurophysiol. 1992;9:456–456.
doi: 10.1097/00004691-199210000-00002
Kunanbayev K, Abibullaev B, Zollanvari A. Data augmentation for p300-based brain-computer interfaces using generative adversarial networks. In 2021 9th International Winter Conference on Brain-Computer Interface (BCI). IEEE; 2021: 1–7.
Abdelghaffar Y, Hashem A, Eldawlatly S. Generative Adversarial Networks for Augmenting EEG Data in P300-based Applications: A Comparative Study. In 2022 IEEE 35th International Symposium on Computer-Based Medical Systems (CBMS). IEEE; 2022: 1–6.
Bhandari V, Londhe ND, Kshirsagar GB. TL-P3GAN: an efficient temporal learning-based Generative Adversarial Network for Precise P300 Signal Generation for P300 spellers. IEEE Trans Cogn Dev Syst 2023.
Zhang R, Zeng Y, Tong L, Shu J, Lu R, Yang K, Li Z, Yan B. Erp-wgan: a data augmentation method for EEG single-trial detection. J Neurosci Methods. 2022;376:109621.
doi: 10.1016/j.jneumeth.2022.109621
Vialatte F-B, Maurice M, Dauwels J, Cichocki A. Steady-state visually evoked potentials: focus on essential paradigms and future perspectives. Prog Neurobiol. 2010;90:418–38.
doi: 10.1016/j.pneurobio.2009.11.005
Aznan NKN, Atapour-Abarghouei A, Bonner S, Connolly JD, Al Moubayed N, Breckon TP. Simulating brain signals: Creating synthetic eeg data via neural-based generative models for improved ssvep classification. In 2019 International joint conference on neural networks (IJCNN). IEEE; 2019: 1–8.
Kwon J, Im C-H. Novel Signal-to-Signal translation method based on StarGAN to generate artificial EEG for SSVEP-based brain-computer interfaces. Expert Syst Appl. 2022;203:117574.
doi: 10.1016/j.eswa.2022.117574
Rosa G, Laitz T, Vargas G, Boccato L. Application of Generative Models for Synthesizing EEG Signals in BCI-SSVEP Systems. Available at SSRN 4385951.
Pan Y, Li N, Zhang Y. Short-time SSVEP data extension by a novel generative adversarial networks based framework. arXiv Preprint arXiv:230105599 2023.
Singh A, Hussain AA, Lal S, Guesgen HW. A comprehensive review on critical issues and possible solutions of motor imagery based electroencephalography brain-computer interface. Sensors. 2021;21:2173.
doi: 10.3390/s21062173
Fahimi F, Dosen S, Ang KK, Mrachacz-Kersting N, Guan C. Generative adversarial networks-based data augmentation for brain–computer interface. IEEE Trans Neural Networks Learn Syst. 2020;32:4039–51.
doi: 10.1109/TNNLS.2020.3016666
Li Z, Yu Y. Improving EEG-based motor imagery classification with conditional Wasserstein GAN. In 2020 International Conference on Image, Video Processing and Artificial Intelligence. SPIE; 2020: 437–443.
Luo T-j, Fan Y, Chen L, Guo G, Zhou C. EEG signal reconstruction using a generative adversarial network with wasserstein distance and temporal-spatial-frequency loss. Front Neuroinformatics. 2020;14:15.
doi: 10.3389/fninf.2020.00015
Zhang K, Xu G, Han Z, Ma K, Zheng X, Chen L, Duan N, Zhang S. Data augmentation for motor imagery signal classification based on a hybrid neural network. Sensors. 2020;20:4485.
doi: 10.3390/s20164485
Habashi AG, Azab AM, Eldawlatly S, Aly GM. Motor Imagery Classification Enhancement using Generative Adversarial Networks for EEG Spectrum Image Generation. In 2023 IEEE 36th International Symposium on Computer-Based Medical Systems (CBMS). IEEE; 2023: 354–359.
Tosato G, Dalbagno CM, Fumagalli F. EEG Synthetic Data Generation Using Probabilistic Diffusion Models. arXiv preprint arXiv:230306068 2023.
Luo Y, Zhu L-Z, Wan Z-Y, Lu B-L. Data augmentation for enhancing EEG-based emotion recognition with deep generative models. J Neural Eng. 2020;17:056021.
doi: 10.1088/1741-2552/abb580
Fu B, Li F, Niu Y, Wu H, Li Y, Shi G. Conditional generative adversarial network for EEG-based emotion fine-grained estimation and visualization. J Vis Commun Image Represent. 2021;74:102982.
doi: 10.1016/j.jvcir.2020.102982
Ledig C, Theis L, Huszár F, Caballero J, Cunningham A, Acosta A, Aitken A, Tejani A, Totz J, Wang Z. Photo-realistic single image super-resolution using a generative adversarial network. In Proceedings of the IEEE conference on computer vision and pattern recognition. 2017: 4681–4690.
Mihajlović V, Grundlehner B, Vullers R, Penders J. Wearable, wireless EEG solutions in daily life applications: what are we missing? IEEE J Biomedical Health Inf. 2014;19:6–21.
doi: 10.1109/JBHI.2014.2328317
Svantesson M, Olausson H, Eklund A, Thordstein M. Virtual EEG-electrodes: convolutional neural networks as a method for upsampling or restoring channels. J Neurosci Methods. 2021;355:109126.
doi: 10.1016/j.jneumeth.2021.109126
Kwon M, Han S, Kim K, Jun SC. Super-resolution for improving EEG spatial resolution using deep convolutional neural network—feasibility study. Sensors. 2019;19:5317.
doi: 10.3390/s19235317
Tang Y, Chen D, Liu H, Cai C, Li X. Deep EEG superresolution via correlating brain structural and functional connectivities. IEEE Trans Cybernetics 2022.
Corley IA, Huang Y. Deep EEG super-resolution: Upsampling EEG spatial resolution with generative adversarial networks. In 2018 IEEE EMBS International Conference on Biomedical & Health Informatics (BHI). IEEE; 2018: 100–103.
Olejniczak P. Neurophysiologic basis of EEG. J Clin Neurophysiol. 2006;23:186–9.
doi: 10.1097/01.wnp.0000220079.61973.6c
Sun R, Sohrabpour A, Worrell GA, He B. Deep neural networks constrained by neural mass models improve electrophysiological source imaging of spatiotemporal brain dynamics. Proceedings of the National Academy of Sciences 2022, 119:e2201128119.
Antoniades A, Spyrou L, Martin-Lopez D, Valentin A, Alarcon G, Sanei S, Took CC. Deep neural architectures for mapping scalp to intracranial EEG. Int J Neural Syst. 2018;28:1850009.
doi: 10.1142/S0129065718500090
Hu M, Chen J, Jiang S, Ji W, Mei S, Chen L, Wang X. E2SGAN: EEG-to-SEEG translation with generative adversarial networks. Front NeuroSci. 2022;16:971829.
doi: 10.3389/fnins.2022.971829
Abdi-Sargezeh B, Oswal A, Sanei S. Mapping Scalp to Intracranial EEG using Generative Adversarial Networks for Automatically Detecting Interictal Epileptiform Discharges. In 2023 IEEE Statistical Signal Processing Workshop (SSP). IEEE; 2023: 710–714.
Qin Z, Li Q. High rate BCI with portable devices based on EEG. Smart Health. 2018;9:115–28.
doi: 10.1016/j.smhl.2018.07.006
Hartmann KG, Schirrmeister RT, Ball T. EEG-GAN: generative adversarial networks for electroencephalograhic (EEG) brain signals. arXiv Preprint arXiv:180601875 2018.
Panwar S, Rad P, Quarles J, Huang Y. Generating EEG signals of an RSVP experiment by a class conditioned wasserstein generative adversarial network. In 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC). IEEE; 2019: 1304–1310.
Aznan NKN, Atapour-Abarghouei A, Bonner S, Connolly JD, Breckon TP: Leveraging synthetic subject invariant EEG signals for zero calibration BCI. In. 2020 25th International Conference on Pattern Recognition (ICPR). IEEE; 2021: 10418–10425.
Xie J, Zhang J, Sun J, Ma Z, Qin L, Li G, Zhou H, Zhan Y. A transformer-based approach combining deep learning network and spatial-temporal information for raw EEG classification. IEEE Trans Neural Syst Rehabil Eng. 2022;30:2126–36.
doi: 10.1109/TNSRE.2022.3194600
Song Y, Yang L, Jia X, Xie L. Common spatial generative adversarial networks based EEG data augmentation for cross-subject brain-computer interface. arXiv Preprint arXiv:210204456 2021.
Panachakel JT, Ramakrishnan AG. Decoding covert speech from EEG-a comprehensive review. Front NeuroSci. 2021;15:392.
doi: 10.3389/fnins.2021.642251
Guo Y, Liu T, Zhang X, Wang A, Wang W. End-to-end translation of human neural activity to speech with a dual–dual generative adversarial network. Knowl Based Syst. 2023;277:110837.
doi: 10.1016/j.knosys.2023.110837
Lee Y-E, Lee S-H, Kim S-H, Lee S-W. Towards voice reconstruction from EEG during imagined speech. In Proceedings of the AAAI Conference on Artificial Intelligence. 2023: 6030–6038.
Kim S, Lee Y-E, Lee S-H, Lee S-W. Diff-E: Diffusion-based Learning for Decoding Imagined Speech EEG. arXiv preprint arXiv:230714389 2023.
Bai Y, Wang X, Cao Y, Ge Y, Yuan C, Shan Y. DreamDiffusion: Generating High-Quality images from Brain EEG signals. arXiv Preprint arXiv:230616934 2023.
Kavasidis I, Palazzo S, Spampinato C, Giordano D, Shah M. Brain2image: Converting brain signals into images. In Proceedings of the 25th ACM international conference on Multimedia. 2017: 1809–1817.
Lan Y-T, Ren K, Wang Y, Zheng W-L, Li D, Lu B-L, Qiu L. Seeing through the brain: Image Reconstruction of Visual Perception from human brain signals. arXiv Preprint arXiv:230802510 2023.
Zeng H, Xia N, Tao M, Pan D, Zheng H, Wang C, Xu F, Zakaria W, Dai G. DCAE: a dual conditional autoencoder framework for the reconstruction from EEG into image. Biomed Signal Process Control. 2023;81:104440.
doi: 10.1016/j.bspc.2022.104440
Mishra R, Sharma K, Jha R, Bhavsar A. NeuroGAN: image reconstruction from EEG signals via an attention-based GAN. Neural Comput Appl. 2023;35:9181–92.
Lu J, Speier W, Hu X, Pouratian N. The effects of stimulus timing features on P300 speller performance. Clin Neurophysiol. 2013;124:306–14.
doi: 10.1016/j.clinph.2012.08.002
Ryan DB, Frye G, Townsend G, Berry D, Mesa-G S, Gates NA, Sellers EW. Predictive spelling with a P300-based brain–computer interface: increasing the rate of communication. Intl J Human–Computer Interact. 2010;27:69–84.
doi: 10.1080/10447318.2011.535754
Zhao WX, Zhou K, Li J, Tang T, Wang X, Hou Y, Min Y, Zhang B, Zhang J, Dong Z. A survey of large language models. arXiv Preprint arXiv:230318223 2023.
Speier W, Arnold C, Pouratian N. Integrating language models into classifiers for BCI communication: a review. J Neural Eng. 2016;13:031002.
doi: 10.1088/1741-2560/13/3/031002
Bansal MA, Sharma DR, Kathuria DM. A systematic review on data scarcity problem in deep learning: solution and applications. ACM Comput Surv (CSUR). 2022;54:1–29.
doi: 10.1145/3502287
Soufineyestani M, Dowling D, Khan A. Electroencephalography (EEG) technology applications and available devices. Appl Sci. 2020;10:7453.
doi: 10.3390/app10217453
Saxena D, Cao J. Generative adversarial networks (GANs) challenges, solutions, and future directions. ACM Comput Surv (CSUR). 2021;54:1–42.
doi: 10.1145/3446374
Ji Z, Lee N, Frieske R, Yu T, Su D, Xu Y, Ishii E, Bang YJ, Madotto A, Fung P. Survey of hallucination in natural language generation. ACM-CSUR. 2023;55:1–38.
Fan J, Fang L, Wu J, Guo Y, Dai Q. From brain science to artificial intelligence. Engineering. 2020;6:248–52.
doi: 10.1016/j.eng.2019.11.012
Song Y, Millidge B, Salvatori T, Lukasiewicz T, Xu Z, Bogacz R. Inferring neural activity before plasticity as a foundation for learning beyond backpropagation. Nat Neurosci 2024:1–11.
Hole KJ, Ahmad S. A thousand brains: toward biologically constrained AI. SN Appl Sci. 2021;3:743.
doi: 10.1007/s42452-021-04715-0