Transparency improves the accuracy of automation use, but automation confidence information does not.
Automation and human cognition
Automation confidence
Automation reliability
Automation transparency
Decision-support systems
Uninhabited vehicle control
Journal
Cognitive research: principles and implications
ISSN: 2365-7464
Titre abrégé: Cogn Res Princ Implic
Pays: England
ID NLM: 101697632
Informations de publication
Date de publication:
08 Oct 2024
08 Oct 2024
Historique:
received:
18
06
2024
accepted:
20
09
2024
medline:
9
10
2024
pubmed:
9
10
2024
entrez:
8
10
2024
Statut:
epublish
Résumé
Increased automation transparency can improve the accuracy of automation use but can lead to increased bias towards agreeing with advice. Information about the automation's confidence in its advice may also increase the predictability of automation errors. We examined the effects of providing automation transparency, automation confidence information, and their potential interacting effect on the accuracy of automation use and other outcomes. An uninhabited vehicle (UV) management task was completed where participants selected the optimal UV to complete missions. Low or high automation transparency was provided, and participants agreed/disagreed with automated advice on each mission. We manipulated between participants whether automated advice was accompanied by confidence information. This information indicated on each trial whether automation was "somewhat" or "highly" confident in its advice. Higher transparency improved the accuracy of automation use, led to faster decisions, lower perceived workload, and increased trust and perceived usability. Providing participant automation confidence information, as compared with not, did not have an overall impact on any outcome variable and did not interact with transparency. Despite no benefit, participants who were provided confidence information did use it. For trials where lower compared to higher confidence information was presented, hit rates decreased, correct rejection rates increased, decision times slowed, and perceived workload increased, all suggestive of decreased reliance on automated advice. Such trial-by-trial shifts in automation use bias and other outcomes were not moderated by transparency. These findings can potentially inform the design of automated decision-support systems that are more understandable by humans in order to optimise human-automation interaction.
Identifiants
pubmed: 39379606
doi: 10.1186/s41235-024-00599-x
pii: 10.1186/s41235-024-00599-x
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
67Subventions
Organisme : Australian Research Council
ID : FT190100812
Organisme : Australian Research Council
ID : DE230100171
Informations de copyright
© 2024. The Author(s).
Références
Bartlett, M. L., & McCarley, J. S. (2017). Benchmarking aided decision making in a signal detection task. Human Factors, 59, 881–900. https://doi.org/10.1177/0018720817700258
doi: 10.1177/0018720817700258
pubmed: 28796974
Bartlett, M. L., & McCarley, J. S. (2019). No effect of cue format on automation dependence in an aided signal detection task. Human Factors, 61, 169–190. https://doi.org/10.1177/0018720818802961
doi: 10.1177/0018720818802961
pubmed: 30335518
Bhaskara, A., Duong, L., Brooks, J., Li, R., McInerney, R., Skinner, M., Pongracic, H., & Loft, S. (2021). Effect of automation transparency in the management of multiple unmanned vehicles. Applied Ergonomics, 90, 103243. https://doi.org/10.1016/j.apergo.2020.103243
doi: 10.1016/j.apergo.2020.103243
pubmed: 32919121
Bhaskara, A., Skinner, M., & Loft, S. (2020). Agent transparency: A review of current theory and evidence. IEEE Transactions on Human-Machine Systems, 50, 215–224. https://doi.org/10.1109/THMS.2020.2965529
doi: 10.1109/THMS.2020.2965529
Bhatt, U., Antoran, J., Zhang, Y., Liao, Q. V., Sattigeri, P., Fogliato, R., Melançon, G., Krishnan, R., Stanley, J., Tickoo, O., Nachman, L., & Xiang, A. (2021). Uncertainty as a form of transparency: Measuring, communicating, and using uncertainty. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society. (pp. 401–413). https://doi.org/10.1145/3461702.3462571
Bisantz, A. M., Marsiglio, S. S., & Munch, J. (2005). Displaying uncertainty: Investigating the effects of display format and specificity. Human Factors, 47, 777–796. https://doi.org/10.1518/001872005775570916
doi: 10.1518/001872005775570916
pubmed: 16553066
Brooke, J. (1996). Sus: A “quick and dirty’usability. Usability evaluation in industry (p. 189). Routledge. https://doi.org/10.1201/9781498710411-35
doi: 10.1201/9781498710411-35
Carter, O. B. J., Loft, S., & Visser, T. A. W. (2024). Meaningful communication but not superficial anthropomorphism facilitates human-automation trust calibration: The Human-Automation Trust Expectation Model (HATEM). Human Factors. https://doi.org/10.1177/00187208231218156
doi: 10.1177/00187208231218156
pubmed: 38041565
Chen, J. Y., Procci, K., Boyce, M., Wright, J., Garcia, A., & Barnes, M. (2014). Situation awareness-based agent transparency (ARL-TR-6905). In U.S. Army Research Laboratory. https://doi.org/10.21236/ada600351
Cohen, J. (1992). Statistical power analysis. Current Directions in Psychological Science, 1, 98–101.
doi: 10.1111/1467-8721.ep10768783
Endsley, M. R., Bolté, B., & Jones, D. G. (2003). Designing for situation awareness: An approach to user-centered design. CRC Press. https://doi.org/10.1201/9780203485088
doi: 10.1201/9780203485088
Endsley, M. R., & Kiris, E. O. (1994). Information presentation for expert systems in future fighters aircraft. The International Journal of Aviation Psychology, 4, 333–348. https://doi.org/10.1207/s15327108ijap0404_3
doi: 10.1207/s15327108ijap0404_3
Gegoff, I., Tatasciore, M., Bowden, V., & Loft, S. (2023). Transparent automated advice to mitigate the impact of variation in automation reliability. Human Factors, 66, 2008–2024. https://doi.org/10.1177/00187208231196738
doi: 10.1177/00187208231196738
pubmed: 37635389
pmcid: 11141097
Hoff, K. A., & Bashir, M. (2015). Trust in automation: Integrating empirical evidence of factors that influence trust. Human Factors, 57, 407–434. https://doi.org/10.1177/0018720814547570
doi: 10.1177/0018720814547570
pubmed: 25875432
Lee, J. D., & See, K. A. (2004). Trust in automation: Designing for appropriate reliance. Human Factors, 46, 50–80. https://doi.org/10.1518/hfes.46.1.50_30392
doi: 10.1518/hfes.46.1.50_30392
pubmed: 15151155
Macmillan, N. A., & Kaplan, H. L. (1985). Detection theory analysis of group data: Estimating sensitivity from average hit and false-alarm rates. Psychological Bulletin, 98, 185. https://doi.org/10.1037/0033-2909.98.1.185
doi: 10.1037/0033-2909.98.1.185
pubmed: 4034817
McGuirl, J. M., & Sarter, N. B. (2006). Supporting trust calibration and the effective use of decision aids by presenting dynamic system confidence information. Human Factors, 48, 656–665. https://doi.org/10.1518/001872006779166334
doi: 10.1518/001872006779166334
pubmed: 17240714
Mercado, J. E., Rupp, M. A., Chen, J. Y., Barnes, M. J., Barber, D., & Procci, K. (2016). Intelligent agent transparency in human–agent teaming for Multi-UxV management. Human Factors, 58, 401–415. https://doi.org/10.1177/0018720815621206
doi: 10.1177/0018720815621206
pubmed: 26867556
Merritt, S. M. (2011). Affective processes in human–automation interactions. Human Factors, 53, 356–370. https://doi.org/10.1177/0018720811411912
doi: 10.1177/0018720811411912
pubmed: 21901933
Moray, N. (2003). Monitoring, complacency, scepticism and eutactic behaviour. International Journal of Industrial Ergonomics, 31, 175–178. https://doi.org/10.1016/S0169-8141(02)00194-4
doi: 10.1016/S0169-8141(02)00194-4
Moray, N., & Inagaki, T. (2000). Attention and complacency. Theoretical Issues in Ergonomics Science, 1, 354–365. https://doi.org/10.1080/14639220052399159
doi: 10.1080/14639220052399159
Mosier, K. L., & Manzey, D. (2019). Humans and automated decision aids: A match made in heaven? In M. Mouloua & P. A. Hancock (Eds.), Human performance in automated and autonomous systems: Current theory and methods (pp. 19–42). CRC Press.
doi: 10.1201/9780429458330-2
Mosier, K. L., Skitka, L. J., Burdick, M. D., & Heers, S. T. (1996). Automation bias, accountability, and verification behaviours. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 40, 204–208. https://doi.org/10.1177/154193129604000413
doi: 10.1177/154193129604000413
NASEM National Academies of Sciences, Engineering, & Medicine. (2022). Human-AI Teaming: State-of-the-Art and Research Needs.
Parasuraman, R., & Manzey, D. H. (2010). Complacency and bias in human use of automation: An attentional integration. Human Factors: THe Journal of the Human Factors and Ergonomics Society, 52, 381–410. https://doi.org/10.1177/0018720810376055
doi: 10.1177/0018720810376055
Parasuraman, R., & Riley, V. (1997). Humans and automation: Use, misuse, disuse, abuse. Human Factors, 39, 230–253. https://doi.org/10.1518/001872097778543886
doi: 10.1518/001872097778543886
Patton, C., & Wickens, C. (2024). The relationship of trust and dependence. Ergonomics. https://doi.org/10.1080/00140139.2024.2342436
doi: 10.1080/00140139.2024.2342436
pubmed: 38725397
Pokam, R., Debernard, S., Chauvin, C., & Langlois, S. (2019). Principles of transparency for autonomous vehicles: First results of an experiment with an augmented reality human–machine interface. Cognition, Technology & Work, 21, 643–656. https://doi.org/10.1007/s10111-019-00552-9
doi: 10.1007/s10111-019-00552-9
Sadler, G., Battiste, H., Ho, N., Hoffmann, L., Johnson, W., Shively, R., Lyons, J., & Smith, D. (2016). Effects of transparency on pilot trust and agreement in the autonomous constrained flight planner. In 2016 IEEE/AIAA 35th Digital Avionics Systems Conference (DASC). (pp. 1–9). IEEE. https://doi.org/10.1109/DASC.2016.7777998
Sargent, R., Walters, B., & Wickens, C. (2023). Meta-analysis qualifying and quantifying the benefits of automation transparency to enhance models of human performance. In International Conference on Human-Computer Interaction. (pp. 243–261). Cham: Springer Nature Switzerland. https://doi.org/10.1007/978-3-031-35596-7_16
Senders, J. W. (1983). Visual Scanning Processes. University of Tilburg Press.
Skraaning, G., & Jamieson, G. A. (2021). Human performance benefits of the automation transparency design principle: Validation and variation. Human Factors, 63, 379–401. https://doi.org/10.1177/0018720819887252
doi: 10.1177/0018720819887252
pubmed: 31834815
Steelman, K. S., McCarley, J. S., & Wickens, C. D. (2011). Modeling the control of attention in visual workspaces. Human Factors, 53, 142–153. https://doi.org/10.1177/0018720811404026
doi: 10.1177/0018720811404026
pubmed: 21702332
Stein, E. S. (1985). Air traffic controller workload: An examination of workload probe (No. DOT/FAA/CT-TN84/24). United States. Department of Transportation. Federal Aviation Administration. William J. Hughes Technical Center.
Stowers, K., Kasdaglis, N., Rupp, M. A., Newton, O. B., Chen, J. Y., & Barnes, M. J. (2020). The IMPACT of agent transparency on human performance. IEEE Transactions on Human-Machine Systems, 50, 245–253. https://doi.org/10.1109/THMS.2020.2978041
doi: 10.1109/THMS.2020.2978041
Strickland, K., Farrell, S., Wilson, M. K., Hutchinson, J., & Loft, S. (2024). How do humans learn about the reliability of automation? Cognitive Research: Principles and Implications, 9, 8. https://doi.org/10.1186/s41235-024-00533-1
doi: 10.1186/s41235-024-00533-1
pubmed: 38361149
Strickland, L., Boag, R. J., Heathcote, A., Bowden, V., & Loft, S. (2023). Automated decision aids: When are they advisors and when do they take control of human decision making? Journal of Experimental Psychology: Applied, 29, 849–868. https://doi.org/10.1037/xap0000463
doi: 10.1037/xap0000463
pubmed: 36877467
Strickland, L., Heathcote, H., Bowden, V., Boag, R. J., Wilson, M. D., Khan, S., & Loft, S. (2021). Inhibitory cognitive control allows automated advice to improve accuracy while minimizing misuse. Psychological Science, 32, 1768–1781. https://doi.org/10.1177/09567976211012676
doi: 10.1177/09567976211012676
pubmed: 34570615
Tatasciore, M., Bowden, V., & Loft, S. (2023). Do concurrent task demands impact the benefit of automation transparency? Applied Ergonomics, 101, 104022. https://doi.org/10.1016/j.apergo.2023.104022
doi: 10.1016/j.apergo.2023.104022
Tatasciore, M., & Loft, S. (2024). Can increased automation transparency mitigate the effects of time pressure on automation use? Applied Ergonomics, 114, 104142. https://doi.org/10.1016/j.apergo.2023.104142
doi: 10.1016/j.apergo.2023.104142
pubmed: 37757606
Tversky, A., & Kahneman, D. (1992). Advances in prospect theory: Cumulative representation of uncertainty. Journal of Risk and Uncertainty, 5, 297–323. https://doi.org/10.1007/BF00122574
doi: 10.1007/BF00122574
Van de Merwe, K., Mallam, S., & Nazir, S. (2022). Agent transparency, situation awareness, mental workload, and operator performance: A systematic literature review. Human Factors, 66, 180. https://doi.org/10.1177/00187208221077804
doi: 10.1177/00187208221077804
pubmed: 35274577
pmcid: 10756021
Van de Merwe, K., Mallam, S., Nazir, S., & Engelhardtsen, Ø. (2024). The influence of agent transparency and complexity on situation awareness, mental workload, and task performance. Journal of Cognitive Engineering and Decision Making, 18, 156–184. https://doi.org/10.1177/1555343424124055
doi: 10.1177/1555343424124055
Wickens, C. D., & Dixon, S. R. (2007). The benefits of imperfect diagnostic automation: A synthesis of the literature. Theoretical Issues in Ergonomics Science, 8, 201–212. https://doi.org/10.1080/14639220500370105
doi: 10.1080/14639220500370105
Wickens, C. D., Sebok, A., Li, H., Sarter, N., & Gacy, A. M. (2015). Using modeling and simulation to predict operator performance and automation-induced complacency with robotic automation: A case study and empirical validation. Human Factors, 57(6), 959–975. https://doi.org/10.1177/0018720814566454
doi: 10.1177/0018720814566454
pubmed: 25850111
Wiczorek, R., & Manzey, D. (2014). Supporting attention allocation in multitask environments: Effects of likelihood alarm systems on trust, behaviour, and performance. Human Factors, 56, 1209–1221. https://doi.org/10.1177/0018720814528534
doi: 10.1177/0018720814528534
pubmed: 25490802
Wiczorek, R., Manzey, D., & Zirk, A. (2014). Benefits of decision-support by likelihood versus binary alarm systems: Does the number of stages make a difference? In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 58, 380–384. https://doi.org/10.1177/1541931214581078
doi: 10.1177/1541931214581078
Wulff, D. U., Mergenthaler-Canseco, M., & Hertwig, R. (2018). A meta-analytic review of two modes of learning and the description-experience gap. Psychological Bulletin, 144(2), 140–176. https://doi.org/10.1037/bul0000115
doi: 10.1037/bul0000115
pubmed: 29239630
Zhang, H., & Maloney, L. T. (2012). Ubiquitous log odds: A common representation of probability and frequency distortion in perception, action, and cognition. Frontiers in Neuroscience, 6, 1. https://doi.org/10.3389/fnins.2012.00001
doi: 10.3389/fnins.2012.00001
pubmed: 22294978
pmcid: 3261445