e-Babylab: An open-source browser-based tool for unmoderated online developmental studies.
Browser-based testing platform
Developmental research
Online studies
Unmoderated testing
Journal
Behavior research methods
ISSN: 1554-3528
Titre abrégé: Behav Res Methods
Pays: United States
ID NLM: 101244316
Informations de publication
Date de publication:
24 Aug 2023
24 Aug 2023
Historique:
accepted:
12
07
2023
medline:
25
8
2023
pubmed:
25
8
2023
entrez:
24
8
2023
Statut:
aheadofprint
Résumé
The COVID-19 pandemic massively changed the context and feasibility of developmental research. This new reality, as well as considerations about sample diversity and naturalistic settings for developmental research, highlights the need for solutions for online studies. In this article, we present e-Babylab, an open-source browser-based tool for unmoderated online studies targeted for young children and babies. e-Babylab offers an intuitive graphical user interface for study creation and management of studies, users, participant data, and stimulus material, with no programming skills required. Various kinds of audiovisual media can be presented as stimuli, and possible measures include webcam recordings, audio recordings, key presses, mouse-click/touch coordinates, and reaction times. An additional feature of e-Babylab is the possibility to administer short adaptive versions of MacArthur-Bates Communicative Development Inventories (Chai et al. Journal of Speech, Language, and Hearing Research, 63, 3488-3500, 2020). Information pages, consent forms, and participant forms are customizable. e-Babylab has been used with a variety of measures and paradigms in over 12 studies with children aged 12 months to 8 years (n = 1516). We briefly summarize some results of these studies to demonstrate that data quality, participant engagement, and overall results are comparable between laboratory and online settings. Finally, we discuss helpful tips for using e-Babylab and present plans for upgrades.
Identifiants
pubmed: 37620744
doi: 10.3758/s13428-023-02200-7
pii: 10.3758/s13428-023-02200-7
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Subventions
Organisme : Norges Forskningsråd
ID : 223265
Organisme : FRGS
ID : FRGS-NFHY0004
Informations de copyright
© 2023. The Author(s).
Références
Ackermann, L., Lo, C. H., Mani, N., & Mayor, J. (2020). Word learning from a tablet app: Toddlers perform better in a passive context. PLoS ONE, 15(12), e0240519. https://doi.org/10.1371/journal.pone.0240519
doi: 10.1371/journal.pone.0240519
pubmed: 33259476
pmcid: 7707543
Anwyl-Irvine, A. L., Massonnié, J., Flitton, A., Kirkham, N., & Evershed, J. K. (2020). Gorilla in our midst: An online behavioral experiment builder. Behavior Research Methods, 52(1), 388–407. https://doi.org/10.3758/s13428-019-01237-x
doi: 10.3758/s13428-019-01237-x
pubmed: 31016684
Bates, D., Mächler, M., Bolker, B., & Walker, S. (2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67(1), 1–48. https://doi.org/10.18637/jss.v067.i01
Bothe, R., Trouillet, L., Elsner, B., & Mani, N. (in prep). The interplay of word and category knowledge on learning.
Chai, J. H., Lo, C. H., & Mayor, J. (2020). A Bayesian-Inspired Item Response Theory-Based Framework to Produce Very Short Versions of MacArthur–Bates Communicative Development Inventories. Journal of Speech, Language, and Hearing Research, 63(10), 3488–3500. https://doi.org/10.1044/2020_JSLHR-20-00361
doi: 10.1044/2020_JSLHR-20-00361
pubmed: 32897770
Chmielewski, M., & Kucker, S. C. (2020). An MTurk Crisis? Shifts in Data Quality and the Impact on Study Results. Social Psychological and Personality Science, 11(4), 464–473. https://doi.org/10.1177/1948550619875149
doi: 10.1177/1948550619875149
de Leeuw, J. R. (2015). jsPsych: A JavaScript library for creating behavioral experiments in a Web browser. Behavior Research Methods, 47(1), 1–12. https://doi.org/10.3758/s13428-014-0458-y
doi: 10.3758/s13428-014-0458-y
pubmed: 24683129
Django Software Foundation. (2019). Django. Retrieved from https://djangoproject.com
Finger, H.., Goecke, C., Diekamp, D., Standvoß, K., and Koenig, P. (2017) LabVanced: A unified Javascript framework for online studies. In Proceedings of the International Conference on Computational Social Science IC2S2, (pp. 1–3), Cologne, Germany
Frank, M. C., Braginsky, M., Yurovsky, D., & Marchman, V. A. (2017). Wordbank: An open repository for developmental vocabulary data. Journal of Child Language, 44(3), 677–694. https://doi.org/10.1017/S0305000916000209
doi: 10.1017/S0305000916000209
pubmed: 27189114
Golinkoff, R. M., Hirsh-Pasek, K., Cauley, K. M., & Gordon, L. (1987). The eyes have it: lexical and syntactic comprehension in a new paradigm. Journal of Child Language, 14(1), 23–45. https://doi.org/10.1017/s030500090001271x
doi: 10.1017/s030500090001271x
pubmed: 3558524
Golinkoff, R. M., Ma, W., Song, L., & Hirsh-Pasek, K. (2013). Twenty-Five Years Using the Intermodal Preferential Looking Paradigm to Study Language Acquisition: What Have We Learned? Perspectives on Psychological Science, 8(3), 316–339. https://doi.org/10.1177/1745691613484936
doi: 10.1177/1745691613484936
pubmed: 26172975
Gosling, S. D., Vazire, S., Srivastava, S., & John, O. P. (2004). Should we trust web-based studies? A comparative analysis of six preconceptions about internet questionnaires. The American psychologist, 59(2), 93–104. https://doi.org/10.1037/0003-066x.59.2.93
doi: 10.1037/0003-066x.59.2.93
pubmed: 14992636
Henninger, F., Shevchenko, Y., Mertens, U. K., Kieslich, P. J., & Hilbig, B. E. (2021). lab.js: A free, open, online study builder. Behavior Research Methods. 1–18. https://doi.org/10.3758/s13428-019-01283-5
Henrich, J., Heine, S. J., & Norenzayan, A. (2010). The weirdest people in the world? Behavioral and Brain Sciences, 33(2–3), 61–83. https://doi.org/10.1017/s0140525x0999152x
doi: 10.1017/s0140525x0999152x
pubmed: 20550733
Hewson, C. M., Laurent, D., & Vogel, C. M. (1996). Proper methodologies for psychological and sociological studies conducted via the Internet. Behavior Research Methods, Instruments, & Computers, 28(2), 186–191. https://doi.org/10.3758/bf03204763
doi: 10.3758/bf03204763
IBM Cloud Education. (2019). Containerization. https://www.ibm.com/cloud/learn/containerization
Kartushina, N., Rosslund, A., & Mayor, J. (2021). Toddlers raised in multi-dialectal families learn words better in accented speech than those raised in monodialectal families. Journal of Child Language, 1–26. https://doi.org/10.1017/S0305000921000520
Kraut, R., Olson, J., Banaji, M., Bruckman, A., Cohen, J., & Couper, M. (2004). Psychological research online: report of Board of Scientific Affairs' Advisory Group on the Conduct of Research on the Internet. American Psychologist, 59(2), 105–117. https://doi.org/10.1037/0003-066x.59.2.105
doi: 10.1037/0003-066x.59.2.105
pubmed: 14992637
Lange, K., Kühn, S., & Filevich, E. (2015). Just Another Tool for Online Studies (JATOS): An easy solution for setup and management of web servers supporting online studies. PLoS ONE, 10(6), e0130834. https://doi.org/10.1371/journal.pone.0130834
doi: 10.1371/journal.pone.0130834
pubmed: 26114751
pmcid: 4482716
Lausberg, H., & Sloetjes, H. (2009). Coding gestural behavior with the NEUROGES-ELAN system. Behavior Research Methods, 41(3), 841–849. https://doi.org/10.3758/brm.41.3.841
doi: 10.3758/brm.41.3.841
pubmed: 19587200
Lewis, J., & Fowler, M. (2014). Microservices. https://martinfowler.com/articles/microservices.html
Lo, C. H., Rosslund, A., Chai, J. H., Mayor, J., & Kartushina, N. (2021). Tablet Assessment of Word Comprehension Reveals Coarse Word Representations in 18–20-month-old Toddlers. Infancy, 26(4), 596–616. https://doi.org/10.1111/infa.12401
doi: 10.1111/infa.12401
pubmed: 33813801
Lourenco, S. F., & Tasimi, A. (2020). No Participant Left Behind: Conducting Science During COVID-19. Trends in Cognitive Sciences, 24(8), 583–584. https://doi.org/10.1016/j.tics.2020.05.003
doi: 10.1016/j.tics.2020.05.003
pubmed: 32451239
pmcid: 7211671
Mani, N., & Huettig, F. (2012). Prediction during language processing is a piece of cake-but only for skilled producers. Journal of Experimental Psychology Human Perception & Performance, 38(4), 843–847. https://doi.org/10.1037/a0029284
doi: 10.1037/a0029284
Mathôt, S., Schreij, D., & Theeuwes, J. (2012). OpenSesame: An open-source, graphical experiment builder for the social sciences. Behavior Research Methods, 44(2), 314–324. https://doi.org/10.3758/s13428-011-0168-7
doi: 10.3758/s13428-011-0168-7
pubmed: 22083660
Mayor, J., & Mani, N. (2019). A short version of the MacArthur–Bates Communicative Development Inventories with high validity. Behavior Research Methods, 51(5), 2248–2255. https://doi.org/10.3758/s13428-018-1146-0
doi: 10.3758/s13428-018-1146-0
pubmed: 30306410
Moriarty, K., Kaliski, B., & Rusch, A. (2017). PKCS# 5: Password-based cryptography specification version 2.1. Internet Eng. Task Force (IETF), 8018, 1–40.
NetMarketShare. (2021). Market Share Statistics for Internet Technologies, from https://bit.ly/2N0JYIm
Nielsen, M., Haun, D., Kärtner, J., & Legare, C. H. (2017). The persistent sampling bias in developmental psychology: A call to action. Journal of Experimental Child Psychology, 162, 31–38. https://doi.org/10.1016/j.jecp.2017.04.017
doi: 10.1016/j.jecp.2017.04.017
pubmed: 28575664
Papoutsaki, A., Sangkloy, P., Laskey, J., Daskalova, N., Huang, J., & Hays, J. (2016). WebGazer: scalable webcam eye tracking using user interactions. In Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence - IJCAI, (pp. 3839–3845). Retrieved from https://par.nsf.gov/biblio/10024076
Peirce, J., Gray, J. R., Simpson, S., MacAskill, M., Höchenberger, R., Sogo, H., Kastman, E., & Lindeløv, J. K. (2019). PsychoPy2: Experiments in behavior made easy. Behavior Research Methods, 51(1), 195–203. https://doi.org/10.3758/s13428-018-01193-y
doi: 10.3758/s13428-018-01193-y
pubmed: 30734206
pmcid: 6420413
Rezlescu, C., Danaila, I., Miron, A., & Amariei, C. (2020). More time for science: Using Testable to create and share behavioral experiments faster, recruit better participants, and engage students in hands-on research. Progress in Brain Research, 253, 243–262. https://doi.org/10.1016/bs.pbr.2020.06.005
doi: 10.1016/bs.pbr.2020.06.005
pubmed: 32771126
Rhodes, M., Rizzo, M. T., Foster-Hanson, E., Moty, K., Leshin, R. A., Wang, M., Benitez, J., & Ocampo, J. D. (2020). Advancing Developmental Science via Unmoderated Remote Research with Children. Journal of Cognition and Development, 1-17. https://doi.org/10.1080/15248372.2020.1797751
Robinson, J., Rosenzweig, C., Moss, A. J., & Litman, L. (2019). Tapped out or barely tapped? Recommendations for how to harness the vast and largely unused potential of the Mechanical Turk participant pool. PLoS ONE, 14(12), e0226394. https://doi.org/10.1371/journal.pone.0226394
doi: 10.1371/journal.pone.0226394
pubmed: 31841534
pmcid: 6913990
Santolin, C., Garcia-Castro, G., Zettersten, M., Sebastian-Galles, N., & Saffran, J. R. (2021). Experience with research paradigms relates to infants’ direction of preference. Infancy, 26(1), 39–46. https://doi.org/10.1111/infa.12372
doi: 10.1111/infa.12372
pubmed: 33111438
Sauter, M., Draschkow, D., & Mack, W. (2020). Building, Hosting and Recruiting: A Brief Introduction to Running Behavioral Experiments Online. Brain Sciences, 10(4). https://doi.org/10.3390/brainsci10040251
Scott, K., & Schulz, L. (2017). Lookit (Part 1): A New Online Platform for Developmental Research. Open Mind, 1, 4–14. https://doi.org/10.1162/OPMI_a_00002
doi: 10.1162/OPMI_a_00002
Sheskin, M., Scott, K., Mills, C. M., Bergelson, E., Bonawitz, E., Spelke, E. S., Li, F.-F., Keil, F. C., Gweon, H., Tenenbaum, J. B., Jara-Ettinger, J., Adolph, K. E., Rhodes, M., Frank, M. C., Mehr, S. A., & Schulz, L. (2020). Online developmental science to foster innovation, access, and impact. Trends in Cognitive Sciences, 24(9), 675–678. https://doi.org/10.1016/j.tics.2020.06.004
doi: 10.1016/j.tics.2020.06.004
pubmed: 32624386
pmcid: 7331515
Sheskin, M., & Keil, F. (2018). TheChildLab.com a video chat platform for developmental research. PsyArXiv.com. https://doi.org/10.31234/osf.io/rn7w5
Simonsen, H. G., Kristoffersen, K. E., Bleses, D., Wehberg, S., & Jørgensen, R. N. (2014). The Norwegian Communicative Development Inventories: Reliability, main developmental trends and gender differences. First Language, 34(1), 3–23. https://doi.org/10.1177/0142723713510997
doi: 10.1177/0142723713510997
Spelke, E. S. (1979). Perceiving bimodally specified events in infancy. Developmental Psychology, 15, 626–636. https://doi.org/10.1037/0012-1649.15.6.626
doi: 10.1037/0012-1649.15.6.626
Stoet, G. (2017). PsyToolkit: A novel web-based method for running online questionnaires and reaction-time experiments. Teaching of Psychology, 44(1), 24–31. https://doi.org/10.1177/0098628316677643
doi: 10.1177/0098628316677643
The OWASP Foundation. (2017). OWASP Top Ten 2017. https://owasp.org/www-project-top-ten/2017/
von Bastian, C. C., Locher, A., & Ruflin, M. (2013). Tatool: A Java-based open-source programming framework for psychological studies. Behavior Research Methods, 45(1), 108–115. https://doi.org/10.3758/s13428-012-0224-y
doi: 10.3758/s13428-012-0224-y
Zaadnoordijk, L., Buckler, H., Cusack, R., Tsuji, S., & Bergmann, C. (2021). A global perspective on testing infants online: Introducing ManyBabies-AtHome. Frontiers in Psychology, 12, 703234. https://doi.org/10.3389/fpsyg.2021.703234