Virtual/augmented reality-based human-machine interface and interaction modes in airport control towers.
Air traffic control
Airport control tower
Augmented reality
Human machine interface
Multimodal interaction
Safety nets
Journal
Scientific reports
ISSN: 2045-2322
Titre abrégé: Sci Rep
Pays: England
ID NLM: 101563288
Informations de publication
Date de publication:
12 06 2024
12 06 2024
Historique:
received:
16
02
2024
accepted:
30
05
2024
medline:
13
6
2024
pubmed:
13
6
2024
entrez:
12
6
2024
Statut:
epublish
Résumé
The concept of an innovative human-machine interface and interaction modes based on virtual and augmented reality technologies for airport control towers has been developed with the aim of increasing the human performances and situational awareness of air traffic control operators. By presenting digital information through see-through head-mounted displays superimposed over the out-of-the-tower view, the proposed interface should stimulate controllers to operate in a head-up position and, therefore, reduce the number of switches between a head-up and a head-down position even in low visibility conditions. This paper introduces the developed interface and describes the exercises conducted to validate the technical solutions developed, focusing on the simulation platform and exploited technologies, to demonstrate how virtual and augmented reality, along with additional features such as adaptive human-machine interface, multimodal interaction and attention guidance, enable a more natural and effective interaction in the control tower. The results of the human-in-the-loop real-time validation exercises show that the prototype concept is feasible from both an operational and technical perspective, the solution proves to support the air traffic controllers in working in a head-up position more than head-down even with low-visibility operational scenarios, and to lower the time to react in critical or alerting situations with a positive impact on the human performances of the user. While showcasing promising results, this study also identifies certain limitations and opportunities for refinement, aimed at further optimising the efficacy and usability of the proposed interface.
Identifiants
pubmed: 38866827
doi: 10.1038/s41598-024-63731-3
pii: 10.1038/s41598-024-63731-3
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
13579Subventions
Organisme : SESAR Joint Undertaking under the European Union's grant agreement
ID : No. 874470
Informations de copyright
© 2024. The Author(s).
Références
Pinska, E. An investigation of the head-up time at tower and ground control positions. In Proceeding of the 5th Eurocontrol Innovative Research Workshop. 81–86 (2006).
Pinska, E. & Tijus, C. Augmented reality technology for control tower analysis of applicability based on the field study. In Proceeding of 1 st CEAS European Air and Space Conference. 573–580 (Citeseer, 2007).
Papenfuss, A. et al. Assessing operational validity of remote tower control in high-fidelity tower simulation. IFAC Proc. Vol. 43, 117–122 (2010).
doi: 10.3182/20100831-4-FR-2021.00022
Papenfuss, A. & Friedrich, M. Head up only-a design concept to enable multiple remote tower operations. In 2016 IEEE/AIAA 35th Digital Avionics Systems Conference (DASC). 1–10 (IEEE, 2016).
Hilburn, B. Head-Down Time in Aerodrome Operations: A Scope Study. (Center for Human Performance Research, 2004).
Hilburn, B. & Jorna, P. G. Workload and air traffic control. In Stress, Workload, and Fatigue. 384–394 (CRC Press, 2000).
Fürstenau, N. Virtual and Remote Control Tower: Research, Design, Development, Validation, and Implementation (2022).
Reuschling, F. & Jakobi, J. Remote AFIS: Development and validation of low-cost remote tower concepts for uncontrolled aerodromes. CEAS Aeronaut. J. 1–17 (2022).
Reisman, R. & Brown, D. Design of augmented reality tools for air traffic control towers. In 6th AIAA Aviation Technology, Integration and Operations Conference (ATIO). 7713 (2006).
Tavanti, M. Augmented reality for tower: using scenarios for describing tower activities. In 2007 IEEE/AIAA 26th Digital Avionics Systems Conference. Vol. 5-A (IEEE, 2007).
Silva, A., Ferreira, E., Laforge, N. & Carvalho, S. The use of surveillance data in augmented reality system to assist control tower operations. In 2015 3rd Experiment International Conference (exp. at’15). 178–182 (IEEE, 2015).
Gürlük, H. Concept of an adaptive augmented vision based assistance system for air traffic control towers. In 2016 IEEE/AIAA 35th Digital Avionics Systems Conference (DASC). 1–10 (IEEE, 2016).
Gürlük, H., Gluchshenko, O., Finke, M., Christoffels, L. & Tyburzy, L. Assessment of risks and benefits of context-adaptive augmented reality for aerodrome control towers. In 2018 IEEE/AIAA 37th Digital Avionics Systems Conference (DASC). 1–10 (IEEE, 2018).
Masotti, N., Bagassi, S. & Crescenzio, F. D. Augmented reality for the control tower: The retina concept. In International Conference on Augmented Reality, Virtual Reality and Computer Graphics. 444–452 (Springer, 2016).
Resilient Synthetic Vision for Advanced Control Tower Air Navigation Service Provision. http://www.retina-atm.eu/index.html (2018).
Bagassi, S., Crescenzio, F. D., Lucchi, F. & Masotti, N. Augmented and virtual reality in the airport control tower. In 30th Congress of the International Council of the Aeronautical Sciences, 30th Congress of the International Council of the Aeronautical Sciences (International Council of the Aeronautical Sciences, 2016).
Ellejmi, M., Bagassi, S. & Persiani, A. Evaluation of augmented reality tools for the provision of tower air traffic control using an ecological interface design. In 2018 Modeling and Simulation Technologies Conference. 2939 (2018).
Bagassi, S. et al. Human-in-the-loop evaluation of an augmented reality based interface for the airport control tower. Comput. Indus. 123, 103291 (2020).
doi: 10.1016/j.compind.2020.103291
Digital Technologies for Tower. https://www.sesarju.eu/projects/dtt (2020).
Project pj05-w2 dtt—digital technologies for tower. https://www.remote-tower.eu/wp/project-pj05-w2/ (2023).
Teutsch, J., Bos, T., van Apeldoorn, M. & Camara, L. Attention guidance for tower ATC using augmented reality devices. In 2022 Integrated Communication, Navigation and Surveillance Conference (ICNS). 1–12 (IEEE, 2022).
Santarelli, R. et al. Towards a digital control tower: The use of augmented reality tools to innovate interaction modes. Sesar Innov. Days 2022 (2022).
Rehmann, J. T., Stein, E. S. & Rosenberg, B. L. Subjective pilot workload assessment. Hum. Factors 25, 297–301 (1983).
doi: 10.1177/001872088302500306
Naval Air Warfare Center Aircraft Division Patuxent River MD. Proceedings for the Annual Symposium and Exhibition on Situational Awareness in the Tactical Air Environment (3rd), held at Piney Point, Maryland, on 2–3 June 1998 (1998).
Lee, K., Kerns, K., Bone, R. & Nickelson, M. Development and validation of the controller acceptance rating scale (cars): Results of empirical research. In Proceedings of the 4th USA/Europe Air Traffic Management R &D Seminar (2001).