Instant interaction driven adaptive gaze control interface.
Journal
Scientific reports
ISSN: 2045-2322
Titre abrégé: Sci Rep
Pays: England
ID NLM: 101563288
Informations de publication
Date de publication:
22 May 2024
22 May 2024
Historique:
received:
24
10
2023
accepted:
16
05
2024
medline:
23
5
2024
pubmed:
23
5
2024
entrez:
22
5
2024
Statut:
epublish
Résumé
Gaze estimation is long been recognised as having potential as the basis for human-computer interaction (HCI) systems, but usability and robustness of performance remain challenging . This work focuses on systems in which there is a live video stream showing enough of the subjects face to track eye movements and some means to infer gaze location from detected eye features. Currently, systems generally require some form of calibration or set-up procedure at the start of each user session. Here we explore some simple strategies for enabling gaze based HCI to operate immediately and robustly without any explicit set-up tasks. We explore different choices of coordinate origin for combining extracted features from multiple subjects and the replacement of subject specific calibration by system initiation based on prior models. Results show that referencing all extracted features to local coordinate origins determined by subject start position enables robust immediate operation. Combining this approach with an adaptive gaze estimation model using an interactive user interface enables continuous operation with the 75th percentile gaze errors of 0.7
Identifiants
pubmed: 38778122
doi: 10.1038/s41598-024-62365-9
pii: 10.1038/s41598-024-62365-9
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
11661Subventions
Organisme : Engineering and Physical Sciences Research Council
ID : EP/W035154/1
Organisme : European Research Council
ID : 319456
Pays : International
Organisme : Medical Research Council
ID : MR/P008712/1
Pays : United Kingdom
Organisme : Wellcome Trust
ID : WT 203148/Z/16/Z
Pays : United Kingdom
Informations de copyright
© 2024. The Author(s).
Références
Hansen, D. W. & Ji, Q. In the eye of the beholder: A survey of models for eyes and gaze. IEEE Trans. Pattern Anal. Mach. Intell. 32, 478–500 (2009).
doi: 10.1109/TPAMI.2009.30
Liu, J., Chi, J., Yang, H. & Yin, X. In the eye of the beholder: A survey of gaze tracking techniques. Pattern Recognit. 108944 (2022).
Zhang, X., Sugano, Y., Fritz, M. & Bulling, A. Mpiigaze: Real-world dataset and deep appearance-based gaze estimation. IEEE Trans. Pattern Anal. Mach. Intell. 41, 162–175 (2017).
doi: 10.1109/TPAMI.2017.2778103
pubmed: 29990057
Su, M.-C. et al. An eye-tracking system based on inner corner-pupil center vector and deep neural network. Sensors 20, 25 (2019).
doi: 10.3390/s20010025
pubmed: 31861512
pmcid: 6983074
Aunsri, N. & Rattarom, S. Novel eye-based features for head pose-free gaze estimation with web camera: New model and low-cost device. Ain Shams Eng. J. 13, 101731 (2022).
doi: 10.1016/j.asej.2022.101731
Zhang, X., Sugano, Y. & Bulling, A. Revisiting data normalization for appearance-based gaze estimation. In Proceedings of the 2018 ACM Symposium on Eye Tracking Research & Applications. 1–9 (2018).
Qian, K. et al. An eye tracking based virtual reality system for use inside magnetic resonance imaging systems. Sci. Rep. 11, 1–17 (2021).
doi: 10.1038/s41598-021-95634-y
Cheng, Y., Bao, Y. & Lu, F. Puregaze: Purifying gaze feature for generalizable gaze estimation. In Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 36. 436–443 (2022).
Xu, M., Wang, H. & Lu, F. Learning a generalized gaze estimator from gaze-consistent feature. In Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 37. 3027–3035 (2023).
Sesma, L., Villanueva, A. & Cabeza, R. Evaluation of pupil center-eye corner vector for gaze estimation using a web cam. In Proceedings of the Symposium on Eye Tracking Research and Applications. 217–220 (2012).
Cheung, Y.-M. & Peng, Q. Eye gaze tracking with a web camera in a desktop environment. IEEE Trans. Hum. Mach. Syst. 45, 419–430 (2015).
doi: 10.1109/THMS.2015.2400442
Cai, H., Yu, H., Zhou, X. & Liu, H. Robust gaze estimation via normalized iris center-eye corner vector. In International Conference on Intelligent Robotics and Applications. 300–309 (Springer, 2016).
Wu, Y.-L., Yeh, C.-T., Hung, W.-C. & Tang, C.-Y. Gaze direction estimation using support vector machine with active appearance model. Multimed. Tools Appl. 70, 2037–2062 (2014).
doi: 10.1007/s11042-012-1220-z
Cerrolaza, J. J., Villanueva, A. & Cabeza, R. Taxonomic study of polynomial regressions applied to the calibration of video-oculographic systems. In Proceedings of the 2008 Symposium on Eye Tracking Research & Applications. 259–266 (2008).
Hornof, A. J. & Halverson, T. Cleaning up systematic error in eye-tracking data by using required fixation locations. Behav. Res. Methods Instrum. Comput. 34, 592–604 (2002).
doi: 10.3758/BF03195487
pubmed: 12564562
Sugano, Y., Matsushita, Y., Sato, Y. & Koike, H. An incremental learning method for unconstrained gaze estimation. In European Conference on Computer Vision. 656–667 (Springer, 2008).
Huang, M. X., Kwok, T. C., Ngai, G., Chan, S. C. & Leong, H. V. Building a personalized, auto-calibrating eye tracker from user interactions. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. 5169–5179 (2016).
Zhang, X., Huang, M. X., Sugano, Y. & Bulling, A. Training person-specific gaze estimators from user interactions with multiple devices. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1–12 (2018).
Sugano, Y., Matsushita, Y. & Sato, Y. Appearance-based gaze estimation using visual saliency. IEEE Trans. Pattern Anal. Mach. Intell. 35, 329–341 (2012).
doi: 10.1109/TPAMI.2012.101
Wang, K., Wang, S. & Ji, Q. Deep eye fixation map learning for calibration-free eye gaze tracking. In Proceedings of the Ninth Biennial ACM Symposium on Eye Tracking Research & Applications. 47–55 (2016).
Hiroe, M., Yamamoto, M. & Nagamatsu, T. Implicit user calibration for gaze-tracking systems using an averaged saliency map around the optical axis of the eye. In Proceedings of the 2018 ACM Symposium on Eye Tracking Research & Applications. 1–5 (2018).
Kang, I. & Malpeli, J. G. Behavioral calibration of eye movement recording systems using moving targets. J. Neurosci. Methods 124, 213–218 (2003).
doi: 10.1016/S0165-0270(03)00019-0
pubmed: 12706852
Pfeuffer, K., Vidal, M., Turner, J., Bulling, A. & Gellersen, H. Pursuit calibration: Making gaze calibration less tedious and more flexible. In Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology. 261–270 (2013).
Blignaut, P. Using smooth pursuit calibration for difficult-to-calibrate participants. J. Eye Mov. Res. 10 (2017).
Land, M. & Tatler, B. Looking and Acting: Vision and Eye Movements in Natural Behaviour (Oxford University Press, 2009).
doi: 10.1093/acprof:oso/9780198570943.001.0001
Sidenmark, L. & Lundström, A. Gaze behaviour on interacted objects during hand interaction in virtual reality for eye tracking calibration. In Proceedings of the 11th ACM Symposium on Eye Tracking Research & Applications. 1–9 (2019).
Shih, S.-W. & Liu, J. A novel approach to 3-D gaze tracking using stereo cameras. IEEE Trans. Syst. Man Cybern. Part B (Cybern.) 34, 234–245 (2004).
doi: 10.1109/TSMCB.2003.811128
Yoo, D. H. & Chung, M. J. A novel non-intrusive eye gaze estimation using cross-ratio under large head motion. Comput. Vis. Image Underst. 98, 25–51 (2005).
doi: 10.1016/j.cviu.2004.07.011
Hansen, D. W., Agustin, J. S. & Villanueva, A. Homography normalization for robust gaze estimation in uncalibrated setups. In Proceedings of the 2010 Symposium on Eye-Tracking Research & Applications. 13–20 (2010).
Kang, J. J., Guestrin, E. D., Maclean, W. J. & Eizenman, M. Simplifying the cross-ratios method of point-of-gaze estimation. In CMBES Proceedings. Vol. 30 (2007).
Coutinho, F. L. & Morimoto, C. H. Free head motion eye gaze tracking using a single camera and multiple light sources. In 2006 19th Brazilian Symposium on Computer Graphics and Image Processing. 171–178 (IEEE, 2006).
Arar, N. M., Gao, H. & Thiran, J.-P. A regression-based user calibration framework for real-time gaze estimation. IEEE Trans. Circuits Syst. Video Technol. 27, 2623–2638 (2016).
doi: 10.1109/TCSVT.2016.2595322
Redmon, J., Divvala, S., Girshick, R. & Farhadi, A. You only look once: Unified, real-time object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 779–788 (2016).
Ou, W.-L., Kuo, T.-L., Chang, C.-C. & Fan, C.-P. Deep-learning-based pupil center detection and tracking technology for visible-light wearable gaze tracking devices. Appl. Sci. 11, 851 (2021).
doi: 10.3390/app11020851
Dwyer, B., Nelson, J., Hansen, T. et al. Roboflow (Version 1.0). https://roboflow.com (Computer Vision Software, 2024).
Martin, D., Malpica, S., Gutierrez, D., Masia, B. & Serrano, A. Multimodality in VR: A survey. ACM Comput. Surv. (CSUR) 54, 1–36 (2022).
doi: 10.1145/3508361
Blignaut, P. Mapping the pupil-glint vector to gaze coordinates in a simple video-based eye tracker. J. Eye Mov. Res. 7 (2014).