EEG-based measurement system for monitoring student engagement in learning 4.0.
Journal
Scientific reports
ISSN: 2045-2322
Titre abrégé: Sci Rep
Pays: England
ID NLM: 101563288
Informations de publication
Date de publication:
07 04 2022
07 04 2022
Historique:
received:
28
09
2021
accepted:
11
03
2022
entrez:
8
4
2022
pubmed:
9
4
2022
medline:
12
4
2022
Statut:
epublish
Résumé
A wearable system for the personalized EEG-based detection of engagement in learning 4.0 is proposed. In particular, the effectiveness of the proposed solution is assessed by means of the classification accuracy in predicting engagement. The system can be used to make an automated teaching platform adaptable to the user, by managing eventual drops in the cognitive and emotional engagement. The effectiveness of the learning process mainly depends on the engagement level of the learner. In case of distraction, lack of interest or superficial participation, the teaching strategy could be personalized by an automatic modulation of contents and communication strategies. The system is validated by an experimental case study on twenty-one students. The experimental task was to learn how a specific human-machine interface works. Both the cognitive and motor skills of participants were involved. De facto standard stimuli, namely (1) cognitive task (Continuous Performance Test), (2) music background (Music Emotion Recognition-MER database), and (3) social feedback (Hermans and De Houwer database), were employed to guarantee a metrologically founded reference. In within-subject approach, the proposed signal processing pipeline (Filter bank, Common Spatial Pattern, and Support Vector Machine), reaches almost 77% average accuracy, in detecting both cognitive and emotional engagement.
Identifiants
pubmed: 35393470
doi: 10.1038/s41598-022-09578-y
pii: 10.1038/s41598-022-09578-y
pmc: PMC8987513
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
5857Informations de copyright
© 2022. The Author(s).
Références
Battro, A. M. & Fischer, K. W. Mind, brain, and education in the digital era. Mind Brain Educ. 6(1), 49 (2012).
doi: 10.1111/j.1751-228X.2011.01137.x
Barrett, R. et al. Social and tactile mixed reality increases student engagement in undergraduate lab activities. J. Chem. Educ. 95(10), 1755 (2018).
doi: 10.1021/acs.jchemed.8b00212
Gan, H. S. et al. Augmented reality experimentation on oxygen gas generation from hydrogen peroxide and bleach reaction. Biochem. Mol. Biol. Educ. 46(3), 245 (2018).
pubmed: 29488323
doi: 10.1002/bmb.21117
Yoon, S. A., Elinich, K., Wang, J., Steinmeier, C. & Tucker, S. Using augmented reality and knowledge-building scaffolds to improve learning in a science museum. Int. J. Comput.-Support. Collab. Learn. 7(4), 519 (2012).
doi: 10.1007/s11412-012-9156-x
Klopp, M. & Abke, J. In 2018 IEEE International Conference on Teaching, Assessment, and Learning for Engineering (TALE) (IEEE, 2018), pp. 871–876.
Mardiana, H. & Daniels, H. K. Technological determinism, new literacies and learning process and the impact towards future learning. Online Submiss. 5(3), 219 (2019).
Lindgren, R. & Johnson-Glenberg, M. Emboldened by embodiment: Six precepts for research on embodied learning and mixed reality. Educ. Res. 42(8), 445 (2013).
doi: 10.3102/0013189X13511661
Anjarichert, L. P., Gross, K., Schuster, K. & Jeschke, S. Learning 4.0: Virtual immersive engineering education. Digit. Univ 2, 51 (2016).
Janssen, D., Tummel, C., Richert, A. & Isenhardt, I. Virtual environments in higher education-immersion as a key construct for learning 4.0. IJAC 9(2), 20 (2016).
Willingham, D. B. A neuropsychological theory of motor skill learning. Psychol. Rev. 105(3), 558 (1998).
pubmed: 9697430
doi: 10.1037/0033-295X.105.3.558
Sailer, U., Flanagan, J. R. & Johansson, R. S. Eye-hand coordination during learning of a novel visuomotor task. J. Neurosci. 25(39), 8833 (2005).
pubmed: 16192373
pmcid: 6725583
doi: 10.1523/JNEUROSCI.2658-05.2005
Bagustari, B. & Santoso, H. in J. Phys. Conf. Ser., vol. 1235 (IOP Publishing, 2019) 1235: 012033
Hu, P. J. H. & Hui, W. Examining the role of learning engagement in technology-mediated learning and its effects on learning effectiveness and satisfaction. Decis. Support Syst. 53(4), 782 (2012).
doi: 10.1016/j.dss.2012.05.014
Rodgers, T. Student engagement in the e-learning process and the impact on their grades. Int. J. Cyber Soc. Educ. 1(2), 143 (2008).
Park, S. Y. Student engagement and classroom variables in improving mathematics achievement. Asia Pac. Educ. Rev. 6(1), 87 (2005).
doi: 10.1007/BF03024970
Lamborn, S., Newmann, F. & Wehlage, G. The significance and sources of student engagement, Student engagement and achievement in American secondary schools pp. 11–39 (1992)
Lutz, A., Slagter, H. A., Dunne, J. D. & Davidson, R. J. Attention regulation and monitoring in meditation. Trends Cogn. Sci. 12(4), 163 (2008).
pubmed: 18329323
pmcid: 2693206
doi: 10.1016/j.tics.2008.01.005
Connell, J.P. & Wellborn, J.G. in Cultural processes in child development: The Minnesota symposia on child psychology, (Psychology Press, 1991), vol. 23, pp. 43–78
Fredricks, J. A., Blumenfeld, P. C. & Paris, A. H. School engagement: Potential of the concept, state of the evidence. Rev. Educ. Res. 74(1), 59 (2004).
doi: 10.3102/00346543074001059
Khedher, A. B. et al. Tracking students’ mental engagement using eeg signals during an interaction with a virtual learning environment. J. Intell. Learn. Syst. Appl. 11(01), 1 (2019).
Debener, S., Minow, F., Emkes, R., Gandras, K. & De Vos, M. How about taking a low-cost, small, and wireless eeg for a walk?. Psychophysiology 49(11), 1617 (2012).
pubmed: 23013047
doi: 10.1111/j.1469-8986.2012.01471.x
Galway, L., McCullagh, P., Lightbody, G., Brennan, C. & Trainor, D. in 2015 IEEE International Conference on Computer and Information Technology; Ubiquitous Computing and Communications; Dependable, Autonomic and Secure Computing; Pervasive Intelligence and Computing (IEEE, 2015), pp. 1554–1559
LaRocco, J., Le, M.D. & Paeng, D.G. A systemic review of available low-cost eeg headsets used for drowsiness detection, Frontiers in neuroinformatics 14 (2020)
Andreessen, L. M., Gerjets, P., Meurers, D. & Zander, T. O. Toward neuroadaptive support technologies for improving digital reading: A passive bci-based assessment of mental workload imposed by text difficulty and presentation speed during reading. User Model. User-Adap. Inter. 31(1), 75 (2021).
doi: 10.1007/s11257-020-09273-5
Pope, A. T., Bogart, E. H. & Bartolome, D. S. Biocybernetic system evaluates indices of operator engagement in automated task. Biol. Psychol. 40(1–2), 187 (1995).
pubmed: 7647180
doi: 10.1016/0301-0511(95)05116-3
Eldenfria, A. & Al-Samarraie, H. Towards an online continuous adaptation mechanism (ocam) for enhanced engagement: An eeg study. Int. J. Hum. Comput. Interact. 35(20), 1960 (2019).
doi: 10.1080/10447318.2019.1595303
Kosmyna, N. & Maes, P. Attentivu: An eeg-based closed-loop biofeedback system for real-time monitoring and improvement of engagement for personalized learning. Sensors 19(23), 5200 (2019).
pmcid: 6929136
doi: 10.3390/s19235200
McLeod, S. Jean piaget’s theory of cognitive development, Simply Psychology pp. 1–9 (2018)
Garland, E. L. & Howard, M. O. Neuroplasticity, psychosocial genomics, and the biopsychosocial paradigm in the 21st century. Health & Social Work 34(3), 191 (2009).
doi: 10.1093/hsw/34.3.191
Kleim, J.A. & Jones, T.A. Principles of experience-dependent neural plasticity: implications for rehabilitation after brain damage (2008)
Markham, J. A. & Greenough, W. T. Experience-driven brain plasticity: Beyond the synapse. Neuron Glia Biol. 1(4), 351 (2004).
pubmed: 16921405
pmcid: 1550735
doi: 10.1017/S1740925X05000219
Kennedy, M. B. Synaptic signaling in learning and memory. Cold Spring Harb. Perspect. Biol. 8(2), a016824 (2016).
pmcid: 4743082
doi: 10.1101/cshperspect.a016824
Owens, M. T. & Tanner, K. D. Teaching as brain changing: Exploring connections between neuroscience and innovative teaching. CBE-Life Sci. Educ. 16(2), fe2 (2017).
pubmed: 28450442
pmcid: 5459260
doi: 10.1187/cbe.17-01-0005
Doidge, N. The brain that changes itself: Stories of personal triumph from the frontiers of brain science (Penguin, 2007)
VanDeWeghe, R. Engaged Learning (Corwin Press, 2009).
Jang, H., Reeve, J. & Deci, E. L. Engaging students in learning activities: It is not autonomy support or structure but autonomy support and structure. J. Educ. Psychol. 102(3), 588 (2010).
doi: 10.1037/a0019682
Alrashidi, O., Phan, H. P. & Ngu, B. H. Academic engagement: An overview of its definitions, dimensions, and major conceptualisations. Int. Educ. Stud. 9(12), 41 (2016).
doi: 10.5539/ies.v9n12p41
Cappella, E., Kim, H. Y., Neal, J. W. & Jackson, D. R. Classroom peer relationships and behavioral engagement in elementary school: The role of social network equity. Am. J. Commun. Psychol. 52(3–4), 367 (2013).
doi: 10.1007/s10464-013-9603-5
Pilotti, M., Anderson, S., Hardy, P., Murphy, P. & Vincent, P. Factors related to cognitive, emotional, and behavioral engagement in the online asynchronous classroom. Int. J. Teach. Learn. Higher Educ. 29(1), 145 (2017).
Silva, A. & Simoes, R. Handbook of Research on Trends in Product Design and Development: Technological and Organizational Perspectives: Technological and Organizational Perspectives (IGI Global, 2010)
Vesga, J. B., Xu, X. & He, H. in 2021 IEEE Virtual Reality and 3D User Interfaces (VR) (IEEE, 2021), pp. 645–652
Rotgans, J. I. & Schmidt, H. G. Cognitive engagement in the problem-based learning classroom. Adv. Health Sci. Educ. 16(4), 465 (2011).
doi: 10.1007/s10459-011-9272-9
Wigfield, A. et al. Role of reading engagement in mediating effects of reading comprehension instruction on reading outcomes. Psychol. Sch. 45(5), 432 (2008).
doi: 10.1002/pits.20307
Helme, S. & Clarke, D. Identifying cognitive engagement in the mathematics classroom. Math. Educ. Res. J. 13(2), 133 (2001).
doi: 10.1007/BF03217103
Chen, P. S. D., Lambert, A. D. & Guidry, K. R. Engaging online learners: The impact of web-based learning technology on college student engagement. Comput. Educ. 54(4), 1222 (2010).
doi: 10.1016/j.compedu.2009.11.008
Jaafar, S., Awaludin, N. S. & Bakar, N. S. in E-proceeding of the Conference on Management and Muamalah (2014), pp. 128–135
Darnell, D. K. & Krieg, P. A. Student engagement, assessed using heart rate, shows no reset following active learning sessions in lectures. PLoS ONE 14(12), e0225709 (2019).
pubmed: 31790461
pmcid: 6886849
doi: 10.1371/journal.pone.0225709
Monkaresi, H., Bosch, N., Calvo, R. A. & D’Mello, S. K. Automated detection of engagement using video-based estimation of facial expressions and heart rate. IEEE Trans. Affect. Comput. 8(1), 15 (2016).
doi: 10.1109/TAFFC.2016.2515084
Peacock, J., Purvis, S. & Hazlett, R. L. Which broadcast medium better drives engagement?: measuring the powers of radio and television with electromyography and skin-conductance measurements. J. Advert. Res. 51(4), 578 (2011).
doi: 10.2501/JAR-51-4-578-585
Berka, C. et al. Eeg correlates of task engagement and mental workload in vigilance, learning, and memory tasks. Aviat. Space Environ. Med. 78(5), B231 (2007).
pubmed: 17547324
Kumar, N. & Michmizos, K. P. in 2020 8th IEEE RAS/EMBS International Conference for Biomedical Robotics and Biomechatronics (BioRob) (IEEE, 2020), pp. 521–526
Park, W. et al. Assessment of cognitive engagement in stroke patients from single-trial eeg during motor rehabilitation. IEEE Trans. Neural Syst. Rehabil. Eng. 23(3), 351 (2014).
pubmed: 25248189
doi: 10.1109/TNSRE.2014.2356472
Khedher, A. B., Jraidi, I. & Frasson, C. in EdMedia+ Innovate Learning (Association for the Advancement of Computing in Education (AACE), 2018), pp. 394–401
Emotiv. https://www.emotiv.com/epoc/
Muse. https://choosemuse.com/
Angrisani, L., Arpaia, P., Donnarumma, F., Esposito, A., Frosolone, M., Improta, G., Moccaldi, N., Natalizio, A. & Parvis, M. in 2020 IEEE International Instrumentation and Measurement Technology Conference (I2MTC) (IEEE, 2020), pp. 1–6
Apicella, A., Arpaia, P., Frosolone, M. & Moccaldi, N. High-wearable eeg-based distraction detection in motor rehabilitation. Sci. Rep. 11(1), 1 (2021).
doi: 10.1038/s41598-021-84447-8
Benlamine, M. S., Dufresne, A., Beauchamp, M. H. & Frasson, C. Bargain: behavioral affective rule-based games adaptation interface–towards emotionally intelligent games: Application on a virtual reality environment for socio-moral development, User Modeling and User-Adapted Interaction pp. 1–35 (2021)
Wang, X. W., Nie, D. & Lu, B. L. Emotional state classification from eeg data using machine learning approach. Neurocomputing 129, 94 (2014).
doi: 10.1016/j.neucom.2013.06.046
Soleymani, M., Asghari-Esfeden, S., Fu, Y. & Pantic, M. Analysis of eeg signals and facial expressions for continuous emotion detection. IEEE Trans. Affect. Comput. 7(1), 17 (2015).
doi: 10.1109/TAFFC.2015.2436926
Zhuang, N., Zeng, Y., Tong, L., Zhang, C., Zhang, H. & Yan, B. Emotion recognition from eeg signals using multidimensional information in emd domain, BioMed research international 2017 (2017)
Jraidi, I., Chaouachi, M. & Frasson, C. A hierarchical probabilistic framework for recognizing learners’ interaction experience trends and emotions, Advances in Human-Computer Interaction 2014 (2014)
Aricò, P. et al. A passive brain-computer interface application for the mental workload assessment on professional air traffic controllers during realistic air traffic control tasks. Prog. Brain Res. 228, 295 (2016).
pubmed: 27590973
doi: 10.1016/bs.pbr.2016.04.021
Wang, S., Gwizdka, J. & Chaovalitwongse, W. A. Using wireless eeg signals to assess memory workload in the [Formula: see text]-back task. IEEE Trans. Hum. Mach. Syst. 46(3), 424 (2015).
doi: 10.1109/THMS.2015.2476818
Jenke, R., Peer, A. & Buss, M. Feature extraction and selection for emotion recognition from eeg. IEEE Trans. Affect. Comput. 5(3), 327 (2014).
doi: 10.1109/TAFFC.2014.2339834
Paas, F., Tuovinen, J. E., Tabbers, H. & Van Gerven, P. W. Cognitive load measurement as a means to advance cognitive load theory. Educ. Psychol. 38(1), 63 (2003).
doi: 10.1207/S15326985EP3801_8
Zhang, X. et al. Evaluating photoplethysmogram as a real-time cognitive load assessment during game playing. Int. J. Hum. Comput. Interact. 34(8), 695 (2018).
doi: 10.1080/10447318.2018.1461763
Das, S., Ghosh, L. & Saha, S. in 2020 IEEE International Conference on Electronics, Computing and Communication Technologies (CONECCT) (IEEE, 2020), pp. 1–6
Mandinach, E. B. & Corno, L. Cognitive engagement variations among students of different ability level and sex in a computer problem solving game. Sex Roles 13(3), 241 (1985).
doi: 10.1007/BF00287914
Ke, F., Xie, K. & Xie, Y. Game-based learning engagement: A theory-and data-driven exploration. Br. J. Edu. Technol. 47(6), 1183 (2016).
doi: 10.1111/bjet.12314
Pan, S. J., Tsang, I. W., Kwok, J. T. & Yang, Q. Domain adaptation via transfer component analysis. IEEE Trans. Neural Networks 22(2), 199 (2010).
pubmed: 21095864
doi: 10.1109/TNN.2010.2091281
Angrisani, L., Arpaia, P., Esposito, A. & Moccaldi, N. A wearable brain–computer interface instrument for augmented reality-based inspection in industry 4.0. IEEE Trans. Instrum. Meas. 69(4), 1530 (2019).
doi: 10.1109/TIM.2019.2914712
Kumar, S., Sharma, R., Sharma, A. & Tsunoda, T. in 2016 international joint conference on neural networks (IJCNN) (IEEE, 2016), pp. 2090–2095
Bentlemsan, M., Zemouri, E. T., Bouchaffra, D., Yahya-Zoubir, B. & Ferroudji, K. in 2014 5th International conference on intelligent systems, modelling and simulation (IEEE, 2014), pp. 235–238
Zheng, W. L., Zhang, Y. Q., Zhu, J. Y. & Lu, B. L. in 2015 international conference on affective computing and intelligent interaction (ACII) (IEEE, 2015), pp. 917–922
Pan, S. J. & Yang, Q. A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 22(10), 1345 (2009).
doi: 10.1109/TKDE.2009.191
Noble, W. S. What is a support vector machine?. Nat. Biotechnol. 24(12), 1565 (2006).
pubmed: 17160063
doi: 10.1038/nbt1206-1565
Ab-medica s.p.a. https://www.abmedica.it/ (2020)
Gaume, A., Dreyfus, G. & Vialatte, F. B. A cognitive brain-computer interface monitoring sustained attentional variations during a continuous task. Cogn. Neurodyn. 13(3), 257 (2019).
pubmed: 31168330
pmcid: 6520431
doi: 10.1007/s11571-019-09521-4
Panda, R., Malheiro, R. & Paiva, R. P. Novel audio features for music emotion recognition. IEEE Trans. Affect. Comput. 11(4), 614 (2018).
doi: 10.1109/TAFFC.2018.2820691
Russell, J. A. A circumplex model of affect. J. Pers. Soc. Psychol. 39(6), 1161 (1980).
doi: 10.1037/h0077714
Vanderhasselt, M. A., Remue, J., Ng, K. K., Mueller, S. C. & De Raedt, R. The regulation of positive and negative social feedback: A psychophysiological study. Cognitive Affect. Behav. Neurosci. 15(3), 553 (2015).
doi: 10.3758/s13415-015-0345-8
Antonenko, P., Paas, F., Grabner, R. & Van Gog, T. Using electroencephalography to measure cognitive load. Educ. Psychol. Rev. 22(4), 425 (2010).
doi: 10.1007/s10648-010-9130-y
Benzing, V., Heinks, T., Eggenberger, N. & Schmidt, M. Acute cognitively engaging exergame-based physical activity enhances executive functions in adolescents. PLoS ONE 11(12), e0167501 (2016).
pubmed: 28030542
pmcid: 5193332
doi: 10.1371/journal.pone.0167501
Delorme, A. & Makeig, S. Eeglab: an open source toolbox for analysis of single-trial eeg dynamics including independent component analysis. J. Neurosci. Methods 134(1), 9 (2004).
pubmed: 15102499
doi: 10.1016/j.jneumeth.2003.10.009
Bjork, R. A. & Kroll, J. F. Desirable difficulties in vocabulary learning. Am. J. Psychol. 128(2), 241 (2015).
pubmed: 26255443
pmcid: 4888598
doi: 10.5406/amerjpsyc.128.2.0241
Abdi, H. & Williams, L. J. Principal component analysis. Wiley Interdiscip. Rev. Comput. Stat. 2(4), 433 (2010).
doi: 10.1002/wics.101
Asensio-Cubero, J., Gan, J. Q. & Palaniappan, R. Multiresolution analysis over graphs for a motor imagery based online bci game. Comput. Biol. Med. 68, 21 (2016).
pubmed: 26599827
doi: 10.1016/j.compbiomed.2015.10.016
Friedman, J., Hastie, T. & Tibshirani, R. et al. The elements of statistical learning, vol. 1 (Springer series in statistics New York, 2001)
Bishop, C. M. Pattern Recognition and Machine Learning (Springer, 2006).
Albawi, S., Mohammed, T. A. & Al-Zawi, S. in 2017 International Conference on Engineering and Technology (ICET) (Ieee, 2017), pp. 1–6
Kim, P. MATLAB Deep Learning 121–147 (Springer, 2017).
doi: 10.1007/978-1-4842-2845-6
Van der Maaten, L. & Hinton, G. Visualizing data using t-sne. J. Mach. Learn. Res. 9(11) (2008)
Bynion, T. M. & Feldner, M. T. Self-assessment manikin. Encyclopedia of personality and individual differences pp. 4654–4656 (2020)
Bradley, M. M. & Lang, P. J. Measuring emotion: The self-assessment manikin and the semantic differential. J. Behav. Ther. Exp. Psychiatry 25(1), 49 (1994).
pubmed: 7962581
doi: 10.1016/0005-7916(94)90063-9
Backs, R. W., da Silva, S. P. & Han, K. A comparison of younger and older adults’ self-assessment manikin ratings of affective pictures. Exp. Aging Res. 31(4), 421 (2005).
pubmed: 16147461
doi: 10.1080/03610730500206808