EEG-based detection of emotional valence towards a reproducible measurement of emotions.
Journal
Scientific reports
ISSN: 2045-2322
Titre abrégé: Sci Rep
Pays: England
ID NLM: 101563288
Informations de publication
Date de publication:
03 11 2021
03 11 2021
Historique:
received:
04
05
2021
accepted:
20
09
2021
entrez:
4
11
2021
pubmed:
5
11
2021
medline:
28
1
2022
Statut:
epublish
Résumé
A methodological contribution to a reproducible Measurement of Emotions for an EEG-based system is proposed. Emotional Valence detection is the suggested use case. Valence detection occurs along the interval scale theorized by the Circumplex Model of emotions. The binary choice, positive valence vs negative valence, represents a first step towards the adoption of a metric scale with a finer resolution. EEG signals were acquired through a 8-channel dry electrode cap. An implicit-more controlled EEG paradigm was employed to elicit emotional valence through the passive view of standardized visual stimuli (i.e., Oasis dataset) in 25 volunteers without depressive disorders. Results from the Self Assessment Manikin questionnaire confirmed the compatibility of the experimental sample with that of Oasis. Two different strategies for feature extraction were compared: (i) based on a-priory knowledge (i.e., Hemispheric Asymmetry Theories), and (ii) automated (i.e., a pipeline of a custom 12-band Filter Bank and Common Spatial Pattern). An average within-subject accuracy of 96.1 %, was obtained by a shallow Artificial Neural Network, while k-Nearest Neighbors allowed to obtain a cross-subject accuracy equal to 80.2%.
Identifiants
pubmed: 34732756
doi: 10.1038/s41598-021-00812-7
pii: 10.1038/s41598-021-00812-7
pmc: PMC8566577
doi:
Types de publication
Journal Article
Research Support, Non-U.S. Gov't
Langues
eng
Sous-ensembles de citation
IM
Pagination
21615Subventions
Organisme : the project "Advanced Virtual Adaptive Technologies e-hEAlth" (AVATEA) POR FESR CAMPANIA 2014/2020
ID : AVATEA CUP. B13D18000130007
Informations de copyright
© 2021. The Author(s).
Références
Kleinginna, P. R. & Kleinginna, A. M. A categorized list of emotion definitions, with suggestions for a consensual definition. Motiv. Emot. 5, 345–379 (1981).
doi: 10.1007/BF00992553
Russell, J. A. A circumplex model of affect. J. Personal. Soc. Psychol. 39, 1161 (1980).
doi: 10.1037/h0077714
Choi, M., Koo, G., Seo, M. & Kim, S. W. Wearable device-based system to monitor a driver’s stress, fatigue, and drowsiness. IEEE Trans. Instrum. Meas. 67, 634–645 (2017).
Mühlbacher-Karrer, S. et al. A driver state detection system–combining a capacitive hand detection sensor with physiological sensors. IEEE Trans. Instrum. Meas. 66, 624–636 (2017).
doi: 10.1109/TIM.2016.2640458
Millard, N. & Hole, L. In the moodie: Using ‘affective widgets’ to help contact centre advisors fight stress. In Affect and Emotion in Human-Computer Interaction, 186–193 (Springer, 2008).
Liu, Y., Sourina, O. & Nguyen, M. K. Real-time EEG-based emotion recognition and its applications. In Transactions on computational science XII, 256–277 (Springer, 2011).
Pop, C. A. et al. Can the social robot probo help children with autism to identify situation-based emotions? a series of single case experiments. Int. J. Hum. Robot. 10, 1350025 (2013).
doi: 10.1142/S0219843613500254
Jones, C. & Sutherland, J. Acoustic emotion recognition for affective computer gaming. In Affect and Emotion in Human-Computer Interaction, 209–219 (Springer, 2008).
Paradiso, S. et al. Cerebral blood flow changes associated with attribution of emotional valence to pleasant, unpleasant, and neutral visual stimuli in a pet study of normal subjects. Am. J. Psychiatry 156, 1618–1629 (1999).
doi: 10.1176/ajp.156.10.1618
pubmed: 10518175
Perdiz, J., Pires, G. & Nunes, U. J. Emotional state detection based on emg and eog biosignals: A short survey. In 2017 IEEE 5th Portuguese Meeting on Bioengineering (ENBENG), 1–4 (IEEE, 2017).
Benovoy, M., Cooperstock, J. R. & Deitcher, J. Biosignals analysis and its application in a performance setting. In Proceedings of the International Conference on Bio-Inspired Systems and Signal Processing, 253–258 (2008).
Liao, L.-D. et al. A novel 16-channel wireless system for electroencephalography measurements with dry spring-loaded sensors. IEEE Trans. Instrum. Meas. 63, 1545–1555 (2014).
doi: 10.1109/TIM.2013.2293222
Chen, Y.-C., Lin, B.-S. & Pan, J.-S. Novel noncontact dry electrode with adaptive mechanical design for measuring shapeeeg in a hairy site. IEEE Trans. Instrum. Meas. 64, 3361–3368 (2015).
doi: 10.1109/TIM.2015.2459531
Rossi, G. B. & Berglund, B. Measurement involving human perception and interpretation. Measurement 44, 815–822 (2011).
doi: 10.1016/j.measurement.2011.01.016
Stevens, S. S. The direct estimation of sensory magnitudes: Loudness. Am. J. Psychol. 69, 1–25 (1956).
pubmed: 13302496
doi: 10.2307/1418112
De Bièvre, P. The 2012 international vocabulary of metrology:"vim". Accred. Quali. Assur. 17, 231–232 (2012).
doi: 10.1007/s00769-012-0885-3
Ekman, P. E. & Davidson, R. J. The nature of emotion: Fundamental questions (Oxford University Press, 1994).
Lang, P. J. International affective picture system (iaps): Affective ratings of pictures and instruction manual. Technical report (2005).
Jenke, R., Peer, A. & Buss, M. Feature extraction and selection for emotion recognition from shapeeeg. IEEE Trans. Affect. Comput. 5, 327–339 (2014).
doi: 10.1109/TAFFC.2014.2339834
Demaree, H. A., Everhart, D. E., Youngstrom, E. A. & Harrison, D. W. Brain lateralization of emotional processing: historical roots and a future incorporating “dominance”. Behav. Cognit. Neurosci. Rev. 4, 3–20 (2005).
Coan, J. A. & Allen, J. J. The state and trait nature of frontal EEG asymmetry in emotion. In The asymmetrical brain (eds Hugdahl, K. & Davidson, R. J.) 565–615 (MIT Press, 2003).
Davidson, R. J. Hemispheric asymmetry and emotion. Approaches Emot. 2, 39–57 (1984).
Bechara, A. et al. Insensitivity to future consequences following damage to human prefrontal cortex. Cognition 50, 1–3 (1994).
doi: 10.1016/0010-0277(94)90018-3
Coan, J. A. & Allen, J. J. Frontal shapeEEG asymmetry and the behavioral activation and inhibition systems. Psychophysiology 40, 106–114 (2003).
pubmed: 12751808
doi: 10.1111/1469-8986.00011
Hagemann, D., Naumann, E., Becker, G., Maier, S. & Bartussek, D. Frontal brain asymmetry and affective style: aconceptual replication. Psychophysiology 35, 372–388 (1998).
pubmed: 9643052
doi: 10.1111/1469-8986.3540372
Coan, J. A. & Allen, J. J. Frontal shapeeeg asymmetry as a moderator and mediator of emotion. Biol. Psychol. 67, 7–50 (2004).
pubmed: 15130524
doi: 10.1016/j.biopsycho.2004.03.002
Wolpaw, J. & Wolpaw, E. W. Brain-computer interfaces: principles and practice (OUP (USA, 2012).
Zeng, H. et al. shapeEEG emotion classification using an improved sincnet-based deep learning model. Brain Sci. 9, 326 (2019).
pmcid: 6895992
doi: 10.3390/brainsci9110326
Luo, Y. et al. shapeEEG-based emotion classification using deep neural network and sparse autoencoder. Front. Syst.Neurosci. 14, 43 (2020).
pubmed: 32982703
pmcid: 7492909
doi: 10.3389/fnsys.2020.00043
Luo, Y. et al. shapeEEG-based emotion classification using spiking neural networks. IEEE Access 8, 46007–46016 (2020).
doi: 10.1109/ACCESS.2020.2978163
Wang, F. et al. Emotion recognition with convolutional neural network and shapeeeg-based efdms. Neuropsychologia 107506, (2020).
Cimtay, Y. & Ekmekcioglu, E. Investigating the use of pretrained convolutional neural network on cross-subject and cross-dataset shapeeeg emotion recognition. Sensors 20, 2034 (2020).
pmcid: 7181114
doi: 10.3390/s20072034
Song, T., Zheng, W., Song, P. & Cui, Z. shapeEEG emotion recognition using dynamical graph convolutional neural networks. IEEE Trans. Affecti. Comput. (2018).
Chen, J., Jiang, D. & Zhang, Y. A hierarchical bidirectional gru model with attention for shapeeeg-based emotion classification. IEEE Access 7, 118530–118540 (2019).
doi: 10.1109/ACCESS.2019.2936817
Soroush, M. Z., Maghooli, K., Setarehdan, S. K. & Nasrabadi, A. M. Emotion classification through nonlinear shapeeeg analysis using machine learning methods. Int. Clin. Neurosci. J. 5, 135 (2018).
doi: 10.15171/icnj.2018.26
Ullah, H. et al. Internal emotion classification using shapeeeg signal with sparse discriminative ensemble. IEEE Access 7, 40144–40153 (2019).
doi: 10.1109/ACCESS.2019.2904400
Chakladar, D. D. & Chakraborty, S. shapeEEG based emotion classification using “correlation based subset selection”. Biol. Inspired Cognit. Architect. 24, 98–106 (2018).
Gonzalez, H. A., Muzaffar, S., Yoo, J. & Elfadel, I. M. Biocnn: A hardware inference engine for shapeeeg-based emotion detection. IEEE Access 8, 140896–140914 (2020).
doi: 10.1109/ACCESS.2020.3012900
Xing, X. et al. Sae+ lstm: A new framework for emotion recognition from multi-channel shapeeeg. Front. Neurorobot. 13, 37 (2019).
pubmed: 31244638
pmcid: 6581731
doi: 10.3389/fnbot.2019.00037
Yang, F., Zhao, X., Jiang, W., Gao, P. & Liu, G. Cross-subject emotion recognition using multi-method fusion from high-dimensional features. Front. Comput. Neurosci. 13, 53 (2019).
pubmed: 31507396
pmcid: 6714862
doi: 10.3389/fncom.2019.00053
Liu, Y. et al. Multi-channel shapeeeg-based emotion recognition via a multi-level features guided capsule network. Comput. Biol. Med. 123, 103927 (2020).
pubmed: 32768036
doi: 10.1016/j.compbiomed.2020.103927
Cui, H. et al. shapeEEG-based emotion recognition using an end-to-end regional-asymmetric convolutional neural network. Knowl.-Based Syst. 205, 106243 (2020).
doi: 10.1016/j.knosys.2020.106243
Hemanth, D. J. shapeEEG signal based modified kohonen neural networks for classification of human mental emotions. J. Artif. Intell. Syst. 2, 1–13 (2020).
doi: 10.33969/AIS.2020.21001
Guo, K. et al. A hybrid fuzzy cognitive map/support vector machine approach for shapeeeg-based emotion classification using compressed sensing. Int. J. Fuzzy Syst. 21, 263–273 (2019).
doi: 10.1007/s40815-018-0567-3
Zheng, W.-L. & Lu, B.-L. Investigating critical frequency bands and channels for shapeeeg-based emotion recognition with deep neural networks. IEEE Trans. Auton. Mental Dev. 7, 162–175 (2015).
doi: 10.1109/TAMD.2015.2431497
Seed-dataset. https://bcmi.sjtu.edu.cn/home/seed/seed.html (2021).
Koelstra, S. et al. Deap: A database for emotion analysis; using physiological signals. IEEE Trans. Affecti. Comput. 3, 18–31 (2011).
doi: 10.1109/T-AFFC.2011.15
Deap-dataset. http://www.eecs.qmul.ac.uk/mmv/datasets/deap/readme.html (2021).
Katsigiannis, S. & Ramzan, N. Dreamer: adatabase for emotion recognition through shapeeeg and ecg signals from wireless low-cost off-the-shelf devices. IEEE J. Biomed. Health inform. 22, 98–107 (2017).
pubmed: 28368836
doi: 10.1109/JBHI.2017.2688239
Dreamer-dataset. https://zenodo.org/record/546113#.YMNN6NUzbIU (2021).
Duan, R.-N., Zhu, J.-Y. & Lu, B.-L. Differential entropy feature for EEG-based emotion classification. In 6th International IEEE/EMBS Conference on Neural Engineering (NER), 81–84 (IEEE, 2013).
Gao, Q. et al. shapeEEG based emotion recognition using fusion feature extraction method. Multimed. Tools Appl. 79, 27057–27074 (2020).
doi: 10.1007/s11042-020-09354-y
Mehmood, R. M. & Lee, H. J. shapeEEG based emotion recognition from human brain using hjorth parameters and svm. Int. J. Bio-Sci. Bio-Technol. 7, 23–32 (2015).
doi: 10.14257/ijbsbt.2015.7.3.03
Dan-Glauser, E. S. & Scherer, K. R. The geneva affective picture database (gaped): a new 730-picture database focusing on valence and normative significance. Behav. Res. Methods 43, 468 (2011).
pubmed: 21431997
doi: 10.3758/s13428-011-0064-1
Taran, S. & Bajaj, V. Emotion recognition from single-channel shapeeeg signals using a two-stage correlation and instantaneous frequency-based filtering method. Comput. Methods Programs Biomed. 173, 157–165 (2019).
pubmed: 31046991
doi: 10.1016/j.cmpb.2019.03.015
Ogino, M. & Mitsukura, Y. A mobile application for estimating emotional valence using a single-channel EEG device. In 2018 57th Annual Conference of the Society of Instrument and Control Engineers of Japan (SICE), 1043–1048 (IEEE, 2018).
Jatupaiboon, N., Pan-ngum, S. & Israsena, P. Emotion classification using minimal EEG channels and frequency bands. In The 2013 10th international joint conference on Computer Science and Software Engineering (JCSSE), 21–24 (IEEE, 2013).
Jalilifard, A., Pizzolato, E. B. & Islam, M. K. Emotion classification using single-channel scalp-EEG recording. In 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), 845–849 (IEEE, 2016).
Petrantonakis, P. C. & Hadjileontiadis, L. J. Emotion recognition from shapeeeg using higher order crossings. IEEE Trans. Inf. Technol. Biomed. 14, 186–197 (2009).
pubmed: 19858033
doi: 10.1109/TITB.2009.2034649
Pandey, P. & Seeja, K. Emotional state recognition with EEG signals using subject independent approach. In Data Science and Big Data Analytics, 117–124 (Springer, 2019).
Ang, A.Q.-X., Yeong, Y. Q. & Wee, W. Emotion classification from shapeeeg signals using time-frequency-dwt features and ann. J. Comput. Commun. 5, 75–79 (2017).
doi: 10.4236/jcc.2017.53009
Yang, C.-J., Fahier, N., He, C.-Y., Li, W.-C. & Fang, W.-C. An ai-edge platform with multimodal wearable physiological signals monitoring sensors for affective computing applications. In 2020 IEEE International Symposium on Circuits and Systems (ISCAS), 1–5 (IEEE, 2020).
Marín-Morales, J. et al. Affective computing in virtual reality: emotion recognition from brain and heartbeat dynamics using wearable sensors. Sci. Rep. 8, 1–15 (2018).
doi: 10.1038/s41598-018-32063-4
Wei, Y., Wu, Y. & Tudor, J. A real-time wearable emotion detection headband based on shapeeeg measurement. Sens. Actuators A: Physical 263, 614–621 (2017).
doi: 10.1016/j.sna.2017.07.012
Arpaia, P., Moccaldi, N., Prevete, R., Sannino, I. & Tedesco, A. A wearable shapeeeg instrument for real-time frontal asymmetry monitoring in worker stress analysis. IEEE Trans. Instrum. Meas. 69, 8335–8343 (2020).
Murugappan, M., Juhari, M. R. B. M., Nagarajan, R. & Yaacob, S. An investigation on visual and audiovisual stimulus based emotion recognition using shapeeeg. Int. J. Medi. Eng. Inform. 1, 342–356 (2009).
doi: 10.1504/IJMEI.2009.022645
Sammler, D., Grigutsch, M., Fritz, T. & Koelsch, S. Music and emotion: electrophysiological correlates of the processing of pleasant and unpleasant music. Psychophysiology 44, 293–304 (2007).
pubmed: 17343712
doi: 10.1111/j.1469-8986.2007.00497.x
Brown, L., Grundlehner, B. & Penders, J. Towards wireless emotional valence detection from EEG. In 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 2188–2191 (IEEE, 2011).
Apicella, A., Arpaia, P., Frosolone, M. & Moccaldi, N. High-wearable shapeeeg-based distraction detection in motor rehabilitation. Sci. Rep. 11, 1–9 (2021).
doi: 10.1038/s41598-021-84447-8
Kumar, S., Sharma, A., Mamun, K. & Tsunoda, T. A deep learning approach for motor imagery EEG signal classification. In 2016 3rd Asia-Pacific World Congress on Computer Science and Engineering (APWC on CSE), 34–39 (IEEE, 2016).
Chin, Z. Y., Ang, K. K., Wang, C., Guan, C. & Zhang, H. Multi-class filter bank common spatial pattern for four-class motor imagery bci. In 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 571–574 (IEEE, 2009).
Thomas, K. P., Guan, C., Lau, C. T., Vinod, A. P. & Ang, K. K. A new discriminative common spatial pattern method for motor imagery brain-computer interfaces. IEEE Trans. Biomed. Eng. 56, 2730–2733 (2009).
pubmed: 19605314
doi: 10.1109/TBME.2009.2026181
Bentlemsan, M., Zemouri, E.-T., Bouchaffra, D., Yahya-Zoubir, B. & Ferroudji, K. Random forest and filter bank common spatial patterns for eeg-based motor imagery classification. In 2014 5th International conference on intelligent systems, modelling and simulation, 235–238 (IEEE, 2014).
Basar, M. D., Duru, A. D. & Akan, A. Emotional state detection based on common spatial patterns of shapeeeg. Signal, Image Video Process. 14, 1–9 (2019).
Yan, M., Lv, Z., Sun, W. & Bi, N. An improved common spatial pattern combined with channel-selection strategy for electroencephalography-based emotion recognition. Med. Engi. Phys. 83, 130–141 (2020).
Kumar, S., Sharma, R., Sharma, A. & Tsunoda, T. Decimation filter with common spatial pattern and fishers discriminant analysis for motor imagery classification. In 2016 international joint conference on neural networks (IJCNN), 2090–2095 (IEEE, 2016).
Hastie, T., Tibshirani, R. & Friedman, J. The elements of Statistical Learning: Data Mining, Inference, and Prediction (Springer, 2009).
Bishop, C. M. Pattern Recognition and Machine Learning (Springer, 2006).
Cybenko, G. Approximation by superpositions of a sigmoidal function. Math. Control Signals Syst. (MCSS) 2, 303–314 (1989).
doi: 10.1007/BF02551274
Kroenke, K. & Spitzer, R. L. The phq-9: a new depression diagnostic and severity measure. Psychiatric Ann. 32, 509–515 (2002).
doi: 10.3928/0048-5713-20020901-06
Braunstein, L. M., Gross, J. J. & Ochsner, K. N. Explicit and implicit emotion regulation: a multi-level framework. Soc. Cognit. Affect. Neurosci. 12, 1545–1557 (2017).
pubmed: 28981910
pmcid: 5647798
doi: 10.1093/scan/nsx096
Kurdi, B., Lozano, S. & Banaji, M. R. Introducing the open affective standardized image set (oasis). Behav. Res. Methods 49, 457–470 (2017).
pubmed: 26907748
doi: 10.3758/s13428-016-0715-3
Ab-medica s.p.a. https://www.abmedica.it/ (2020).
Texasinstrument-ads1298. https://www.ti.com/lit/ds/symlink/ads1296r.pdf (2020-02-28).
Radüntz, T., Scouten, J., Hochmuth, O. & Meffert, B. Eeg artifact elimination by extraction of ica-component features using image processing algorithms. J. Neurosci. Methods 243, 84–93 (2015).
pubmed: 25666892
doi: 10.1016/j.jneumeth.2015.01.030
Lotte, F., Congedo, M., Lécuyer, A., Lamarche, F. & Arnaldi, B. A review of classification algorithms for shapeeeg-based brain-computer interfaces. J. Neural Eng. 4, R1 (2007).
pubmed: 17409472
doi: 10.1088/1741-2560/4/2/R01
Cortes, C. & Vapnik, V. Support-vector networks.. Mach. Learn. 20, 273–297 (1995).
doi: 10.1007/BF00994018
Krogh, A. & Hertz, J. A simple weight decay can improve generalization. In Moody, J., Hanson, S. & Lippmann, R. P. (eds.) Advances in Neural Information Processing Systems, vol. 4 (Morgan-Kaufmann, 1992). https://proceedings.neurips.cc/paper/1991/file/8eefcfdf5990e441f0fb6f3fad709e21-Paper.pdf .
Hamadicharef, B. et al. Learning eeg-based spectral-spatial patterns for attention level measurement. In 2009 IEEE International Symposium on Circuits and Systems, 1465–1468 (IEEE, 2009).