OpenMATB: A Multi-Attribute Task Battery promoting task customization, software extensibility and experiment replicability.
Aircraft piloting
Experiment replicability
Microworld
Software extensibility
Task battery
Task customization
Journal
Behavior research methods
ISSN: 1554-3528
Titre abrégé: Behav Res Methods
Pays: United States
ID NLM: 101244316
Informations de publication
Date de publication:
10 2020
10 2020
Historique:
pubmed:
7
3
2020
medline:
9
1
2021
entrez:
7
3
2020
Statut:
ppublish
Résumé
OpenMATB is an open-source variant of the Multi-Attribute Task Battery (MATB) and is available under a free software license. MATB consists of a set of tasks representative of those performed in aircraft piloting. It is used, in particular, to study the effect of automation on decision-making, mental workload, and vigilance. Since the publication of MATB 20 years ago, the subject of automation has grown considerably in importance. After introducing the task battery, this article highlights three main requirements for an up-to-date implementation of MATB. First, there is a need for task customization, to make it possible to change the values, appearance or integrated components (such as rating scales) of the tasks. Second, researchers need software extensibility to enable them to integrate specific features, such as synchronization with psychophysiological devices. Third, to achieve experiment replicability, it is necessary that the source code and the scenario files are easily available and auditable. In the present paper, we explain how these aspects are implemented in OpenMATB by presenting the software architecture and features, while placing special emphasis on the crucial role of the plugin system and the simplicity of the format used in the script files. Finally, we present a number of general trends for the future study of automation in human factors research and ergonomics.
Identifiants
pubmed: 32140999
doi: 10.3758/s13428-020-01364-w
pii: 10.3758/s13428-020-01364-w
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
1980-1990Références
Aarts, A.A., Anderson, J.E., Anderson, C.J., Attridge, P.R., Attwood, A., Axt, J., … Zuni, K. (2015). Estimating the reproductibility of psychological science. Science, 349(6251).
Arnegard, R. J. (1991). Operator Strategies Under Varying Conditions of Workload. NASA Report n°4385.
Chambers, C. D., & Brown, M. (2003). Timing accuracy under Microsoft Windows revealed through external chronometry. Behavior Research Methods, Instruments, & Computers, 35(1), 96-108.
doi: 10.3758/BF03195501
Comstock Jr, J. R. & Arnegard, R. J. (1992). The multi-attribute task battery for human operator workload and strategic behavior research. (NASA-TM-104174). Washington: National Aeronautics and Space Administration.
Dalmaijer, E.S., Mathôt, S., & Van der Stigchel, S. (2014). PyGaze: an open-source, cross-platform toolbox for minimal-effort programming of eye tracking experiments. Behavior Research Methods, 46, 913-921. doi: https://doi.org/10.3758/s13428-013-0422-2
doi: 10.3758/s13428-013-0422-2
pubmed: 24258321
Gazes, Y., Rakitin, B.C., Steffener, J., Habeck, C., Bufferfield, B., Basner, R.C., Ghez, C., & Stern, Y. (2012). Dual-tasking alleviated sleep deprivation disruption in visuomotor tracking: An fMRI study. Brain and Cognition, 78(3), 248-256.
doi: 10.1016/j.bandc.2012.01.004
Hancock, P.A., & Winge, B. (1988). Strategic control of response efficiency (SCORE). Technical Report TRL-NASA-8804, Institute of Safety and Systems Management, Los Angeles. CA.
Hart, S.G. & Staveland, L.E. (1988). Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In P. A. Hancock and N. Meshkati (Eds.), Human mental workload (139-183). Amsterdam: North Holland Press.
doi: 10.1016/S0166-4115(08)62386-9
Hoogeboom, P. J. (2002). Off-line synchronization of measurements based on a common pseudorandom binary signal. Behavior Research Methods, Instruments, & Computers, 35(3), 384-390.
doi: 10.3758/BF03195515
Jian, J.-Y., Bisantz, A. M., & Drury, C. G. (2000). Foundations for an Empirically Determined Scale of Trust in Automated Systems. International Journal of Cognitive Ergonomics, 4(1), 53-71.
doi: 10.1207/S15327566IJCE0401_04
King, T. A., Keifer, K. S., & Hart, S. G. (1989). Description and application of a versatile research task Window/PANES. Paper presented at the Human Factors Society Conference, Denver, CO.
Lawrence, R. (2004). The space efficiency of XML. Information and Software Technology, 46, 753-759.
doi: 10.1016/j.infsof.2004.02.003
Miller, W. D. (2010). The U.S. Air Force-developed Adaptation Of The Multi-attribute Task Battery For The Assessment Of Human Operator Workload And Strategic Behavior. Interim report AFRL-RH-WP-TR-2010-0133.
Molloy, R., & Parasuraman, R. (1996). Monitoring an automated system for a single failure: Vigilance and task complexity effects. Human Factors, 38, 311-322
doi: 10.1177/001872089606380211
Munafò, M. R., Nosek, B. A., Bishop, D. V. M., Button, K. S., Chambers, C. D., Percie Du Sert, N., … Ioannidis, J. P. A. (2017). A manifesto for reproducible science. Nature Human Behaviour, 1(21).
Navarro (2018). A state of science on highly automated driving. Theoretical Issues in Ergonomics Science.
Navarro, J., Heuveline, L., Avril, E., & Cegarra, J. (2020). Human-Machine Interactions and task complexity influence on automation selection and use. Ergonomics.
Papadelis, C., Kourtidou-Papadeli, C., Bamidis, P., & Albani, M. (2007). Effects of imagery training on cognitive performance and use of physiological measures as an assessment tool of mental effort. Brain and Cognition, 64(1), 74-85.
doi: 10.1016/j.bandc.2007.01.001
Parasuraman, R., Molloy, R., & Singh, I. L. (1993). Performance consequences of automation-induced ‘complacency’. The International Journal of Aviation Psychology, 3(1), 1-23.
doi: 10.1207/s15327108ijap0301_1
Peirce, J. W., Gray, J. R., Simpson, S., MacAskill, M. R., Höchenberger, R., Sogo, H., Kastman, E., Lindeløv, J. (2019). PsychoPy2: experiments in behavior made easy. Behavior Research Methods https://doi.org/10.3758/s13428-018-01193-y
Plant, R. R. (2015). A reminder on millisecond timing accuracy and potential replication failure in computer-based psychology experiments: An open letter. Behavior Research Methods, 48(1), 408-411.
doi: 10.3758/s13428-015-0577-0
Popper, K (1934). The Logic of Scientific Discovery. Routledge: London.
Ryffel, C.P., Muehlethaler, C.M., Huber, S.M., & Elfering, A. (2019). Eye tracking as a debriefing tool in upset prevention and recovery training (UPRT) for general aviation pilots. Ergonomics, 62(2), 319-329.
doi: 10.1080/00140139.2018.1501093
Santiago-Espada, Y., Myer, R. R., Latorella, K. A., & Comstock, J. R. (2011). The Multi-attribute Task Battery II (MATB-II): Software For Human Performance And Workload Research: A User’s Guide. NASA Technical Memorandum 217164.
Singh, I. L., Molloy, R., & Parasuraman, R. (1997). Automation-related monitoring inefficiency: The role of display location. International Journal of Human-Computer Studies, 46, 17–30.
doi: 10.1006/ijhc.1996.0081
Thanoon, M. I., Zein-Sabatto, M. S., & McCurry, C. D. (2017). Multi-Attribute Task Battery for Human-Machine Teaming. Paper presented at the International Conference on Advances on Applied Cognitive Computing, Las Vegas, USA.
Venthur, B., & Blankertz, B. (2012). Mushu, a free-and open source BCI signal acquisition, written in python. In Engineering in Medicine and Biology Society (EMBC), 2012 Annual International Conference of the IEEE. doi: 10.1109/EMBC.2012.6346296, (Vol. 2012 pp. 1786–1788): IEEE.