Validation of an open source, remote web-based eye-tracking method (WebGazer) for research in early childhood.
Journal
Infancy : the official journal of the International Society on Infant Studies
ISSN: 1532-7078
Titre abrégé: Infancy
Pays: United States
ID NLM: 100890607
Informations de publication
Date de publication:
18 Oct 2023
18 Oct 2023
Historique:
revised:
25
09
2023
received:
19
01
2023
accepted:
30
09
2023
medline:
18
10
2023
pubmed:
18
10
2023
entrez:
18
10
2023
Statut:
aheadofprint
Résumé
Measuring eye movements remotely via the participant's webcam promises to be an attractive methodological addition to in-person eye-tracking in the lab. However, there is a lack of systematic research comparing remote web-based eye-tracking with in-lab eye-tracking in young children. We report a multi-lab study that compared these two measures in an anticipatory looking task with toddlers using WebGazer.js and jsPsych. Results of our remotely tested sample of 18-27-month-old toddlers (N = 125) revealed that web-based eye-tracking successfully captured goal-based action predictions, although the proportion of the goal-directed anticipatory looking was lower compared to the in-lab sample (N = 70). As expected, attrition rate was substantially higher in the web-based (42%) than the in-lab sample (10%). Excluding trials based on visual inspection of the match of time-locked gaze coordinates and the participant's webcam video overlayed on the stimuli was an important preprocessing step to reduce noise in the data. We discuss the use of this remote web-based method in comparison with other current methodological innovations. Our study demonstrates that remote web-based eye-tracking can be a useful tool for testing toddlers, facilitating recruitment of larger and more diverse samples; a caveat to consider is the larger drop-out rate.
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Subventions
Organisme : NIH HHS
ID : NIH 1R01HD083312
Pays : United States
Organisme : NIH HHS
ID : NIH 1R15HD099706
Pays : United States
Informations de copyright
© 2023 The Authors. Infancy published by Wiley Periodicals LLC on behalf of International Congress of Infant Studies.
Références
Apache Software Foundation. (2012). Apache HTTP server. Version 2.4) (Computer Software). https://apache.org/
Bacon, D., Weaver, H., & Saffran, J. (2021). A framework for online experimenter-moderated looking-time studies assessing infants' linguistic knowledge. Frontiers in Psychology, 12. Article 703839. https://doi.org/10.3389/fpsyg.2021.703839
Bánki, A., de Eccher, M., Falschlehner, L., Hoehl, S., & Markova, G. (2022). Comparing online webcam-and laboratory-based eye-tracking for the assessment of infants’ audio-visual synchrony perception. Frontiers in Psychology, 12. Article 733933. https://doi.org/10.3389/fpsyg.2021.733933
Bogdan, P. C., Dolcos, S., Buetti, S., Lleras, A., & Dolcos, F. (2023). Investigating the suitability of online eye tracking for psychological research: Evidence from comparisons with in-person data using emotion-attention interaction tasks. Behavior Research Methods. https://doi.org/10.3758/s13428-023-02143-z
Brooks, M. E., Kristensen, K., van Benthem, K. J., Magnusson, A., Berg, C. W., Nielsen, A., Skaug, H. J., Maechler, M., & Bolker, B. M. (2017). glmmTMB balances speed and flexibility among packages for zero-inflated Generalized Linear Mixed Modeling. The R Journal, 9(2), 378-400. https://doi.org/10.32614/RJ-2017-066
Byers-Heinlein, K., Bergmann, C., Davies, C., Frank, M. C., Hamlin, J. K., Kline, M., Kominsky, J. F., Kosie, J. E., Lew-Williams, C., Liu, L., Mastroberardino, M., Singh, L., Waddell, C. P. G., Zettersten, M., & Soderstrom, M. (2020). Building a collaborative psychological science: Lessons learned from ManyBabies 1. Canadian Psychology/Psychologie Canadienne, 61(4), 349-363. https://doi.org/10.1037/cap0000216
Callaghan, T., Moll, H., Rakoczy, H., Warneken, F., Liszkowski, U., Behne, T., & Tomasello, M. (2011). Early social cognition in three cultural contexts: III. Individual studies. Monographs of the Society for Research in Child Development, 76(2), 34-104. https://doi.org/10.1111/j.1540-5834.2011.00606.x
Canonical Ltd. (2018). Ubuntu. Version 18.04 LTS) (Computer Software).
Chuey, A., Asaba, M., Bridgers, S., Carrillo, B., Dietz, G., Garcia, T., Leonard, J. A., Liu, S., Merrick, M., Radwan, S., Stegall, J., Velez, N., Woo, B., Wu, Y., Zhou, X. J., Frank, M. C., & Gweon, H. (2021). Moderated online data-collection for developmental research: Methods and replications. Frontiers in Psychology, 12. Article 734398. https://doi.org/10.3389/fpsyg.2021.734398
Chuey, A., Boyce, V., Cao, A., & Frank, M. C. (2022). Conducting developmental research online vs. in-person: A meta-analysis. PsyArXiv. https://doi.org/10.31234/osf.io/qc6fw
Courage, M. L., Frizzell, L. M., Walsh, C. S., & Smith, M. (2021). Toddlers using tablets: They engage, play, and learn. Frontiers in Psychology, 12. Article 564479. https://doi.org/10.3389/fpsyg.2021.564479
Dalrymple, K. A., Manner, M. D., Harmelink, K. A., Teska, E. P., & Elison, J. T. (2018). An examination of recording accuracy and precision from eye tracking data from toddlerhood to adulthood. Frontiers in Psychology, 9. Article 803. https://doi.org/10.3389/fpsyg.2018.00803
de Leeuw, J. R. (2015). jsPsych: A JavaScript library for creating behavioral experiments in a Web browser. Behavior Research Methods, 47(1), 1-12. https://doi.org/10.3758/s13428-014-0458-y
Erel, Y., Shannon, K. A., Chu, J., Scott, K. M., Kline Struhl, M., Cao, P., Tan, X., Hart, P., Raz, G., Piccolo, S., Mei, C., Potter, C., Jaffe-Dax, S., Lew-Williams, C., Tenenbaum, J., Fairchild, K., Bermano, A., & Liu, S. (2022). iCatcher+: Robust and automated annotation of infant's and young children's gaze direction from videos collected in laboratory, field, and online studies. PsyArXiv. https://doi.org/10.31234/osf.io/up97k
Falck-Ytter, T., Gredebäck, G., & von Hofsten, C. (2006). Infants predict other people's action goals. Nature Neuroscience, 9(7), 878-879. https://doi.org/10.1038/nn1729
Fernald, A., Zangl, R., Portillo, A. L., & Marchman, V. A. (2008). Looking while listening: Using eye movements to monitor spoken language comprehension by infants and young children. In I. A. Sekerina, E. Fernandez, & H. Clahsen (Eds.), Developmental Psycholinguistics: On-line methods in children’s language processing (pp. 97-135). John Benjamins.
Finger, H., Goeke, C., Diekamp, D., Standvoß, K., & König, P. (2017). LabVanced: A unified JavaScript framework for online studies [conference paper]. International Conference on Computational Social Science IC2S2S. (Germany).
Frank, M. C., Bergelson, E., Bergmann, C., Cristia, A., Floccia, C., Gervain, J., Hamlin, J. K., Hannon, E. E., Kline, M., Levelt, C., Lew-Williams, C., Nazzi, T., Panneton, R., Rabagliati, H., Soderstrom, M., Sullivan, J., Waxman, S., & Yurovsky, D. (2017). A collaborative approach to infant research: Promoting reproducibility, best practices, and theory-building. Infancy, 22(4), 421-435. https://doi.org/10.1111/infa.12182
GazeRecorder (2010). GazeRecorder (Computer Software) https://gazerecorder.com/
Holmqvist, K., Örbom, S. L., Hooge, I. T. C., Niehorster, D. C., Alexander, R. G., Andersson, R., Benjamins, J. S., Blignaut, P., Brouwer, A.-M., Chuang, L. L., Dalrymple, K. A., Drieghe, D., Dunn, M. J., Ettinger, U., Fiedler, S., Foulsham, T., van der Geest, J. N., Hansen, D. W., Hutton, S. B., …, & Hessels, R. S. (2023). Eye tracking: Empirical foundations for a minimal reporting guideline. Behavior Research Methods, 55(1), 364-416. https://doi.org/10.3758/s13428-021-01762-8
Hutt, S., & D'Mello, S. K. (2022). Evaluating calibration-free webcam-based eye tracking for gaze-based user modeling. In Proceedings of the 2022 international conference on multimodal interaction (pp. 224-235).
Kominsky, J. F., Begus, K., Bass, I., Colantonio, J., Leonard, J. A., Mackey, A. P., & Bonawitz, E. (2021). Organizing the methodological toolbox: Lessons learned from implementing developmental methods online. Frontiers in Psychology, 12. Article 702710. https://doi.org/10.3389/fpsyg.2021.702710
Leshin, R., Leslie, S.-J., & Rhodes, M. (2021). Does it matter how we speak about social kinds? A large, pre-registered, online experimental study of how language shapes the development of essentialist beliefs. Child Development, 92(4), e531-e547. https://doi.org/10.1111/cdev.13527
Levin, D. E., & Rosenquest, B. (2001). The increasing role of electronic toys in the lives of infants and toddlers: Should we be concerned? Contemporary Issues in Early Childhood, 2(2), 242-247. https://doi.org/10.2304/ciec.2001.2.2.9
Lewandowska, B. (2019). RealEye eye-tracking system technology whitepaper. Retrieved https://support.realeye.io/realeye-accuracy/ 19 December 2022.
Liszkowski, U., Carpenter, M., & Tomasello, M. (2007). Pointing out new news, old news, and absent referents at 12 months of age. Developmental Science, 10(2), F1-F7. https://doi.org/10.1111/j.1467-7687.2006.00552.x
Lo, C. H., Mani, N., Kartushina, N., Mayor, J., & Hermes, J. (2021). e-Babylab: an open-source browser-based tool for unmoderated online developmental studies. PsyArXiv. https://doi.org/10.31234/osf.io/u73sy
Luo, Y., & Baillargeon, R. (2007). Do 12.5-month-old infants consider what objects others can see when interpreting their actions? Cognition, 105(3), 489-512. https://doi.org/10.1016/j.cognition.2006.10.007
Nelson, C. M., & Oakes, L. M. (2021). “May I grab your attention?”: An investigation into infants' visual preferences for handled objects using Lookit as an online platform for data collection. Frontiers in Psychology, 12. Article 3866. https://doi.org/10.3389/fpsyg.2021.733218
Nisbett, R. E., & Miyamoto, Y. (2005). The influence of culture: Holistic versus analytic perception. Trends in Cognitive Sciences, 9(10), 467-473. https://doi.org/10.1016/j.tics.2005.08.004
Oakes, L. M. (2012). Advances in eye tracking in infancy research. Infancy, 17(1), 1-8. https://doi.org/10.1111/j.1532-7078.2011.00101.x
Oliver, B. R., & Pike, A. (2021). Introducing a novel online observation of parenting behavior: Reliability and validation. Parenting, 21(2), 168-183. https://doi.org/10.1080/15295192.2019.1694838
Ozkan, A. (2018). Using eye-tracking methods in infant memory research. The Journal of Neurobehavioral Sciences, 5, 62-66.
Papoutsaki, A., Sangkloy, P., Laskey, J., Daskalova, N., Huang, J., & Hays, J. (2016). WebGazer: Scalable webcam eye tracking using user interactions. In Proceedings of the 25th international joint conference on artificial intelligence (pp. 3839-3845). IJCAI).
Prein, J. C., Bohn, M., Kalinke, S., & Haun, D. B. M. (2022). Tango: A reliable, open-source, browser-based task to assess individual differences in gaze understanding in 3 to 5-year-old children and adults. PsyArXiv. https://doi.org/10.31234/osf.io/vghw8
R Core Team. (2021). R: A language and environment for statistical computing. R Foundation for Statistical Computing. http://www.R-project.org/
Rhodes, M., Rizzo, M. T., Foster-Hanson, E., Moty, K., Leshin, R. A., Wang, M., Benitez, J., & Ocampo, J. D. (2020). Advancing developmental science via unmoderated remote research with children. Journal of Cognition and Development, 21(4), 477-493. https://doi.org/10.1080/15248372.2020.1797751
Schidelko, L. P., Schünemann, B., Rakoczy, H., & Proft, M. (2021). Online testing yields the same results as lab testing: A validation study with the false belief task. Frontiers in Psychology, 12. Article 703238. https://doi.org/10.3389/fpsyg.2021.703238
Schneegans, T., Bachman, M. D., Huettel, S. A., & Heekeren, H. (2021). Exploring the potential of online webcam-based eye tracking in decision-making research and influence factors on data quality. PsyArXiv. https://doi.org/10.31234/osf.io/zm3us
Schuwerk, T., Kampis, D., Baillargeon, R., Biro, S., Bohn, M., Byers-Heinlein, K., Dörrenberg, S., Fisher, C., Franchin, L., Fulcher, T., Garbisch, I., Geraci, A., Grosse Wiesmann, C., Hamlin, K., Haun, D. B. M., Hepach, R., Hunnius, S., Hyde, D. C., Karman, P., …, & rakoczy, h. (2022). Action anticipation based on an agent's epistemic state in toddlers and adults. PsyArXiv. https://doi.org/10.31234/osf.io/x4jbm
Scott, K., & Schulz, L. (2017). Lookit (Part 1): A new online platform for developmental research. Open Mind, 1(1), 4-14. https://doi.org/10.1162/OPMI_a_00002
Semmelmann, K., & Weigelt, S. (2018). Online webcam-based eye tracking in cognitive science: A first look. Behavior Research Methods, 50(2), 451-465. https://doi.org/10.3758/s13428-017-0913-7
Senju, A., Southgate, V., White, S., & Frith, U. (2009). Mindblind eyes: An absence of spontaneous theory of mind in Asperger syndrome. Science, 325(5942), 883-885. https://doi.org/10.1126/science.1176170
Sheskin, M., Scott, K., Mills, C. M., Bergelson, E., Bonawitz, E., Spelke, E. S., Fei-Fei, L., Keil, F. C., Gweon, H., Tenenbaum, J. B., Jara-Ettinger, J., Adolph, K. E., Rhodes, M., Frank, M. C., Mehr, S. A., & Schulz, L. (2020). Online developmental science to foster innovation, access, and impact. Trends in Cognitive Sciences, 24(9), 675-678. https://doi.org/10.1016/j.tics.2020.06.004
Singh, L., Cristia, A., Karasik, L. B., Rajendra, S. J., & Oakes, L. (2021). Diversity and Representation in Infant Research: Barriers and bridges towards a globalized science of infant development. PsyArXiv. https://doi.org/10.31234/osf.io/hgukc
Southgate, V., Senju, A., & Csibra, G. (2007). Action anticipation through attribution of false belief by 2-year-olds. Psychological Science, 18(7), 587-592. https://doi.org/10.1111/j.1467-9280.2007.01944.x
Steffan, A., & Müller, T. (2021). ManyKeys. Version 1.0. Computer Software) https://github.com/adriansteffan/manykeys/tree/bed46cdaf3cb8a578c6277eff669b0abb36c3a26
Su, I. A., & Ceci, S. (2021). Zoom Developmentalists”: Home-based videoconferencing developmental research during COVID-19. PsyArXiv. https://doi.org/10.31234/osf.io/nvdy6
The PHP Group. (2020). PHP. Version 8.0) (Computer Software).
Valliappan, N., Dai, N., Steinberg, E., He, J., Rogers, K., Ramachandran, V., Xu, P., Shojaeizadeh, M., Guo, L., Kohlhoff, K., & Navalpakkam, V. (2020). Accelerating eye movement research via accurate and affordable smartphone eye tracking. Nature Communications, 11(1), 4553. Article 4553. https://doi.org/10.1038/s41467-020-18360-5
Venker, C. E., & Kover, S. T. (2015). An open conversation on using eye-gaze methods in studies of neurodevelopmental disorders. Journal of Speech, Language, and Hearing Research, 58(6), 1719-1732. https://doi.org/10.1044/2015_JSLHR-L-14-0304
Visser, I., Bergmann, C., Byers-Heinlein, K., Dal Ben, R., Duch, W., Forbes, S., Franchin, L., Frank, M. C., Geraci, A., Hamlin, J. K., Kaldy, Z., Kulke, L., Laverty, C., Lew-Williams, C., Mateu, V., Mayor, J., Moreau, D., Nomikou, I., Schuwerk, T., … Zettersten, M. (2022). Improving the generalizability of infant psychological research: The ManyBabies model. Behavioral and Brain Sciences, 45, e35. Article e35. https://doi.org/10.1017/S0140525X21000455
Wang, Y., Alpcetin, B., Zhu, J., Buyurucu, G., Sancar, B. H., Kaya, M. E., Dresel, M., Exner, A., Hamlin, J. K., Havron, N., Henderson, A., Martin, A., Partridge, T. T., Schuwerk, T., Shainy, M. R., Su, Y., Tsang, C. K. A., Uzefovsky, F., Wong, T. T.-Y., … Lucca, K. (2023). Individual differences in infants' social evaluations across cultures: A spin-off project of many babies. CEO Conference on Cognitive Development. [Poster Presentation]. The 13th Budapest https://osf.io/jp532
Wass, S. V. (2016). The use of eye-tracking with infants and children. In J. Prior & J. Van Herwegen (Eds.), Practical research with children (1st ed., pp. 24-45). Routledge. https://doi.org/10.4324/9781315676067
Wass, S. V., Smith, T. J., & Johnson, M. H. (2013). Parsing eye-tracking data of variable quality to provide accurate fixation duration estimates in infants and adults. Behavior Research Methods, 45(1), 229-250. https://doi.org/10.3758/s13428-012-0245-6
Werchan, D. M., Thomason, M. E., & Brito, N. H. (2022). Owlet: An automated, open-source method for infant gaze tracking using smartphone and webcam recordings. Behavior Research Methods, 55(6), 3149-3163. https://doi.org/10.3758/s13428-022-01962-w
Wong, A. Y., Bryck, R. L., Baker, R. S., Hutt, S., & Mills, C. (2023). Using a webcam based eye-tracker to understand students’ thought patterns and reading behaviors in neurodivergent classrooms. In LAK23: 13th international learning analytics and knowledge conference (pp. 453-463).
Xu, P., Ehinger, K. A., Zhang, Y., Finkelstein, A., Kulkarni, S. R., & Xiao, J. (2015). TurkerGaze: Crowdsourcing saliency with webcam based eye tracking. arXiv http://arxiv.org/abs/1504.06755
Yang, X., & Krajbich, I. (2021). Webcam-based online eye-tracking for behavioral research. Judgment and Decision Making, 16(6), 1485-1505. https://doi.org/10.1017/S1930297500008512
Yoder, P. J., Lloyd, B. P., & Symons, F. J. (2018). Observational measurement of behavior (2nd ed.). Paul H. Brookes.
Zaadnoordijk, L., Buckler, H., Cusack, R., Tsuji, S., & Bergmann, C. (2021). A global perspective on testing infants online: Introducing ManyBabies-AtHome. Frontiers in Psychology, 12. Article 703234. https://doi.org/10.3389/fpsyg.2021.703234
Zeng, G., Simpson, E. A., & Paukner, A. (2023). Maximizing valid eye-tracking data in human and macaque infants by optimizing calibration and adjusting areas of interest. Behavior Research Methods. https://doi.org/10.3758/s13428-022-02056-3